{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DweYe9FcbMK_"
   },
   "source": [
    "##### Copyright 2018 The TensorFlow Authors.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:22.574945Z",
     "iopub.status.busy": "2022-08-31T05:39:22.574663Z",
     "iopub.status.idle": "2022-08-31T05:39:22.578812Z",
     "shell.execute_reply": "2022-08-31T05:39:22.578258Z"
    },
    "id": "AVV2e0XKbJeX"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sZfSvVcDo6GQ"
   },
   "source": [
    "# 加载文本"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "giK0nMbZFnoR"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>     <a target=\"_blank\" href=\"https://tensorflow.google.cn/tutorials/load_data/text\"><img src=\"https://tensorflow.google.cn/images/tf_logo_32px.png\"> 在 TensorFlow.org 上查看</a>   </td>\n",
    "  <td>     <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/zh-cn/tutorials/load_data/text.ipynb\"><img src=\"https://tensorflow.google.cn/images/colab_logo_32px.png\">在 Google Colab 中运行</a>\n",
    "</td>\n",
    "  <td>     <a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/tutorials/load_data/text.ipynb\"><img src=\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\">在 GitHub 上查看源代码</a>\n",
    "</td>\n",
    "  <td>     <a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/zh-cn/tutorials/load_data/text.ipynb\"><img src=\"https://tensorflow.google.cn/images/download_logo_32px.png\">下载笔记本</a>\n",
    "</td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dwlfPb11GH8J"
   },
   "source": [
    "本教程演示了两种加载和预处理文本的方法。\n",
    "\n",
    "- 首先，您将使用 Keras 效用函数和预处理层。这包括用于将数据转换为 `tf.data.Dataset` 的 `tf.keras.utils.text_dataset_from_directory` 和用于数据标准化、词例化和向量化的 `tf.keras.layers.TextVectorization`。如果您是 TensorFlow 新手，则应当从这些开始。\n",
    "- 然后，您将使用 `tf.data.TextLineDataset` 等较低级别的效用函数来加载文本文件，并使用 [TensorFlow Text](https://tensorflow.google.cn/text) API（如 `text.UnicodeScriptTokenizer` 和 `text.case_fold_utf8`）来预处理数据以实现粒度更细的控制。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:22.583166Z",
     "iopub.status.busy": "2022-08-31T05:39:22.582652Z",
     "iopub.status.idle": "2022-08-31T05:39:48.133487Z",
     "shell.execute_reply": "2022-08-31T05:39:48.132459Z"
    },
    "id": "sa6IKWvADqH7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow-text==2.8.*\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tensorflow_text-2.8.2-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (4.9 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow<2.9,>=2.8.0\r\n",
      "  Downloading tensorflow-2.8.2-cp39-cp39-manylinux2010_x86_64.whl (498.0 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tensorflow-hub>=0.8.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-text==2.8.*) (0.12.0)\r\n",
      "Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (0.26.0)\r\n",
      "Requirement already satisfied: grpcio<2.0,>=1.24.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.49.0rc1)\r\n",
      "Requirement already satisfied: termcolor>=1.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.1.0)\r\n",
      "Requirement already satisfied: wrapt>=1.11.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.14.1)\r\n",
      "Requirement already satisfied: libclang>=9.0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (14.0.6)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorboard<2.9,>=2.8\r\n",
      "  Downloading tensorboard-2.8.0-py3-none-any.whl (5.8 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: opt-einsum>=2.3.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (3.3.0)\r\n",
      "Requirement already satisfied: setuptools in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (65.3.0)\r\n",
      "Requirement already satisfied: flatbuffers>=1.12 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (2.0.7)\r\n",
      "Collecting keras<2.9,>=2.8.0rc0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading keras-2.8.0-py2.py3-none-any.whl (1.4 MB)\r\n",
      "Requirement already satisfied: absl-py>=0.4.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.2.0)\r\n",
      "Requirement already satisfied: typing-extensions>=3.6.6 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (4.3.0)\r\n",
      "Requirement already satisfied: h5py>=2.9.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (3.7.0)\r\n",
      "Requirement already satisfied: six>=1.12.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.16.0)\r\n",
      "Requirement already satisfied: protobuf<3.20,>=3.9.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (3.19.4)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow-estimator<2.9,>=2.8\r\n",
      "  Downloading tensorflow_estimator-2.8.0-py2.py3-none-any.whl (462 kB)\r\n",
      "Requirement already satisfied: astunparse>=1.6.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.6.3)\r\n",
      "Requirement already satisfied: numpy>=1.20 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.23.2)\r\n",
      "Requirement already satisfied: gast>=0.2.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (0.4.0)\r\n",
      "Requirement already satisfied: google-pasta>=0.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (0.2.0)\r\n",
      "Requirement already satisfied: keras-preprocessing>=1.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.1.2)\r\n",
      "Requirement already satisfied: wheel<1.0,>=0.23.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from astunparse>=1.6.0->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (0.37.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (0.6.1)\r\n",
      "Requirement already satisfied: google-auth<3,>=1.6.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (2.11.0)\r\n",
      "Requirement already satisfied: markdown>=2.6.8 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (3.4.1)\r\n",
      "Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (0.4.6)\r\n",
      "Requirement already satisfied: werkzeug>=0.11.15 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (2.2.2)\r\n",
      "Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.8.1)\r\n",
      "Requirement already satisfied: requests<3,>=2.21.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (2.28.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: cachetools<6.0,>=2.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (5.2.0)\r\n",
      "Requirement already satisfied: rsa<5,>=3.1.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (4.9)\r\n",
      "Requirement already satisfied: pyasn1-modules>=0.2.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (0.3.0rc1)\r\n",
      "Requirement already satisfied: requests-oauthlib>=0.7.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.3.1)\r\n",
      "Requirement already satisfied: importlib-metadata>=4.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (4.12.0)\r\n",
      "Requirement already satisfied: idna<4,>=2.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (3.3)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (2022.6.15)\r\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (1.26.12)\r\n",
      "Requirement already satisfied: charset-normalizer<3,>=2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (2.1.1)\r\n",
      "Requirement already satisfied: MarkupSafe>=2.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from werkzeug>=0.11.15->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (2.1.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: zipp>=0.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (3.8.1)\r\n",
      "Requirement already satisfied: pyasn1<0.6.0,>=0.4.6 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (0.5.0rc2)\r\n",
      "Requirement already satisfied: oauthlib>=3.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard<2.9,>=2.8->tensorflow<2.9,>=2.8.0->tensorflow-text==2.8.*) (3.2.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Installing collected packages: tensorflow-estimator, keras, tensorboard, tensorflow, tensorflow-text\r\n",
      "  Attempting uninstall: tensorflow-estimator\r\n",
      "    Found existing installation: tensorflow-estimator 2.10.0rc0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling tensorflow-estimator-2.10.0rc0:\r\n",
      "      Successfully uninstalled tensorflow-estimator-2.10.0rc0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: keras\r\n",
      "    Found existing installation: keras 2.10.0rc1\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling keras-2.10.0rc1:\r\n",
      "      Successfully uninstalled keras-2.10.0rc1\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: tensorboard\r\n",
      "    Found existing installation: tensorboard 2.10.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling tensorboard-2.10.0:\r\n",
      "      Successfully uninstalled tensorboard-2.10.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: tensorflow\r\n",
      "    Found existing installation: tensorflow 2.10.0rc3\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling tensorflow-2.10.0rc3:\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      Successfully uninstalled tensorflow-2.10.0rc3\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully installed keras-2.8.0 tensorboard-2.8.0 tensorflow-2.8.2 tensorflow-estimator-2.8.0 tensorflow-text-2.8.2\r\n"
     ]
    }
   ],
   "source": [
    "!pip install \"tensorflow-text==2.8.*\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:48.138239Z",
     "iopub.status.busy": "2022-08-31T05:39:48.137741Z",
     "iopub.status.idle": "2022-08-31T05:39:50.669088Z",
     "shell.execute_reply": "2022-08-31T05:39:50.668349Z"
    },
    "id": "baYFZMW_bJHh"
   },
   "outputs": [],
   "source": [
    "import collections\n",
    "import pathlib\n",
    "\n",
    "import tensorflow as tf\n",
    "\n",
    "from tensorflow.keras import layers\n",
    "from tensorflow.keras import losses\n",
    "from tensorflow.keras import utils\n",
    "from tensorflow.keras.layers import TextVectorization\n",
    "\n",
    "import tensorflow_datasets as tfds\n",
    "import tensorflow_text as tf_text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Az-d_K5_HQ5k"
   },
   "source": [
    "## 示例 1：预测 Stack Overflow 问题的标签\n",
    "\n",
    "作为第一个示例，您将从 Stack Overflow 下载一个编程问题的数据集。每个问题（*“How do I sort a dictionary by value?”*）都会添加一个标签（`Python`、`CSharp`、`JavaScript` 或 `Java`）。您的任务是开发一个模型来预测问题的标签。这是多类分类的一个示例，多类分类是一种重要且广泛适用的机器学习问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tjC3yLa5IjP7"
   },
   "source": [
    "### 下载并探索数据集\n",
    "\n",
    "首先，使用 `tf.keras.utils.get_file` 下载 Stack Overflow 数据集，然后探索目录结构："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:50.674032Z",
     "iopub.status.busy": "2022-08-31T05:39:50.673579Z",
     "iopub.status.idle": "2022-08-31T05:39:52.503419Z",
     "shell.execute_reply": "2022-08-31T05:39:52.502683Z"
    },
    "id": "8ELgzA6SHTuV"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/stack_overflow_16k.tar.gz\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  16384/6053168 [..............................] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "6053888/6053168 [==============================] - 0s 0us/step\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "6062080/6053168 [==============================] - 0s 0us/step\n"
     ]
    }
   ],
   "source": [
    "data_url = 'https://storage.googleapis.com/download.tensorflow.org/data/stack_overflow_16k.tar.gz'\n",
    "\n",
    "dataset_dir = utils.get_file(\n",
    "    origin=data_url,\n",
    "    untar=True,\n",
    "    cache_dir='stack_overflow',\n",
    "    cache_subdir='')\n",
    "\n",
    "dataset_dir = pathlib.Path(dataset_dir).parent"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:52.507463Z",
     "iopub.status.busy": "2022-08-31T05:39:52.507145Z",
     "iopub.status.idle": "2022-08-31T05:39:52.516666Z",
     "shell.execute_reply": "2022-08-31T05:39:52.516005Z"
    },
    "id": "jIrPl5fUH2gb"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[PosixPath('/tmp/.keras/train'),\n",
       " PosixPath('/tmp/.keras/stack_overflow_16k.tar.gz'),\n",
       " PosixPath('/tmp/.keras/test'),\n",
       " PosixPath('/tmp/.keras/README.md')]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(dataset_dir.iterdir())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:52.520144Z",
     "iopub.status.busy": "2022-08-31T05:39:52.519574Z",
     "iopub.status.idle": "2022-08-31T05:39:52.523890Z",
     "shell.execute_reply": "2022-08-31T05:39:52.523331Z"
    },
    "id": "fEoV7YByJoWQ"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[PosixPath('/tmp/.keras/train/python'),\n",
       " PosixPath('/tmp/.keras/train/javascript'),\n",
       " PosixPath('/tmp/.keras/train/csharp'),\n",
       " PosixPath('/tmp/.keras/train/java')]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_dir = dataset_dir/'train'\n",
    "list(train_dir.iterdir())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3mxAN17MhEh0"
   },
   "source": [
    "`train/csharp`、`train/java`、`train/python` 和 `train/javascript` 目录包含许多文本文件，每个文件都是一个 Stack Overflow 问题。\n",
    "\n",
    "打印示例文件并检查数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:52.527575Z",
     "iopub.status.busy": "2022-08-31T05:39:52.527008Z",
     "iopub.status.idle": "2022-08-31T05:39:52.531013Z",
     "shell.execute_reply": "2022-08-31T05:39:52.530437Z"
    },
    "id": "Go1vTSGdJu08"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "why does this blank program print true x=true.def stupid():.    x=false.stupid().print x\n",
      "\n"
     ]
    }
   ],
   "source": [
    "sample_file = train_dir/'python/1755.txt'\n",
    "\n",
    "with open(sample_file) as f:\n",
    "  print(f.read())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "deWBTkpJiO7D"
   },
   "source": [
    "### 加载数据集\n",
    "\n",
    "接下来，您将从磁盘加载数据并将其准备成适合训练的格式。为此，您将使用 `tf.keras.utils.text_dataset_from_directory` 效用函数来创建带标签的 `tf.data.Dataset`。如果您是 `tf.data` 新手，它是用于构建输入流水线的强大工具集合。（要了解更多信息，请参阅 [tf.data：构建 TensorFlow 输入流水线](../../guide/data.ipynb)指南。）\n",
    "\n",
    "`tf.keras.utils.text_dataset_from_directory` API 需要如下目录结构：\n",
    "\n",
    "```\n",
    "train/\n",
    "...csharp/\n",
    "......1.txt\n",
    "......2.txt\n",
    "...java/\n",
    "......1.txt\n",
    "......2.txt\n",
    "...javascript/\n",
    "......1.txt\n",
    "......2.txt\n",
    "...python/\n",
    "......1.txt\n",
    "......2.txt\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Dyl6JTAjlbQV"
   },
   "source": [
    "运行机器学习实验时，最佳做法是将数据集拆成三份：[训练](https://developers.google.com/machine-learning/glossary#training_set)、[验证](https://developers.google.com/machine-learning/glossary#validation_set)和[测试](https://developers.google.com/machine-learning/glossary#test-set)。\n",
    "\n",
    "Stack Overflow 数据集已经拆分为训练集和测试集，但缺少验证集。\n",
    "\n",
    "通过使用 `tf.keras.utils.text_dataset_from_directory` 并将 `validation_split` 设置为 `0.2`（即 20%），使用训练数据的 80:20 拆分创建验证集："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:52.534652Z",
     "iopub.status.busy": "2022-08-31T05:39:52.534130Z",
     "iopub.status.idle": "2022-08-31T05:39:55.881231Z",
     "shell.execute_reply": "2022-08-31T05:39:55.880473Z"
    },
    "id": "qqyliMw8N-az"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 8000 files belonging to 4 classes.\n",
      "Using 6400 files for training.\n"
     ]
    }
   ],
   "source": [
    "batch_size = 32\n",
    "seed = 42\n",
    "\n",
    "raw_train_ds = utils.text_dataset_from_directory(\n",
    "    train_dir,\n",
    "    batch_size=batch_size,\n",
    "    validation_split=0.2,\n",
    "    subset='training',\n",
    "    seed=seed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DMI_gPLfloD7"
   },
   "source": [
    "正如前面的单元输出所示，训练文件夹中有 8,000 个样本，您将使用其中的 80%（即 6,400 个）进行训练。稍后您将学习到，可以通过将 `tf.data.Dataset` 直接传递给 `Model.fit` 来训练模型。\n",
    "\n",
    "首先，遍历数据集并打印出一些样本来感受一下数据。\n",
    "\n",
    "注：为了增加分类问题的难度，数据集作者将编程问题中出现的单词 *Python*、*CSharp*、*JavaScript* 或 *Java* 替换为 *blank* 一词。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:55.885902Z",
     "iopub.status.busy": "2022-08-31T05:39:55.885262Z",
     "iopub.status.idle": "2022-08-31T05:39:55.913898Z",
     "shell.execute_reply": "2022-08-31T05:39:55.913198Z"
    },
    "id": "_JMTyZ6Glt_C"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Question:  b'\"my tester is going to the wrong constructor i am new to programming so if i ask a question that can be easily fixed, please forgive me. my program has a tester class with a main. when i send that to my regularpolygon class, it sends it to the wrong constructor. i have two constructors. 1 without perameters..public regularpolygon().    {.       mynumsides = 5;.       mysidelength = 30;.    }//end default constructor...and my second, with perameters. ..public regularpolygon(int numsides, double sidelength).    {.        mynumsides = numsides;.        mysidelength = sidelength;.    }// end constructor...in my tester class i have these two lines:..regularpolygon shape = new regularpolygon(numsides, sidelength);.        shape.menu();...numsides and sidelength were declared and initialized earlier in the testing class...so what i want to happen, is the tester class sends numsides and sidelength to the second constructor and use it in that class. but it only uses the default constructor, which therefor ruins the whole rest of the program. can somebody help me?..for those of you who want to see more of my code: here you go..public double vertexangle().    {.        system.out.println(\"\"the vertex angle method: \"\" + mynumsides);// prints out 5.        system.out.println(\"\"the vertex angle method: \"\" + mysidelength); // prints out 30..        double vertexangle;.        vertexangle = ((mynumsides - 2.0) / mynumsides) * 180.0;.        return vertexangle;.    }//end method vertexangle..public void menu().{.    system.out.println(mynumsides); // prints out what the user puts in.    system.out.println(mysidelength); // prints out what the user puts in.    gotographic();.    calcr(mynumsides, mysidelength);.    calcr(mynumsides, mysidelength);.    print(); .}// end menu...this is my entire tester class:..public static void main(string[] arg).{.    int numsides;.    double sidelength;.    scanner keyboard = new scanner(system.in);..    system.out.println(\"\"welcome to the regular polygon program!\"\");.    system.out.println();..    system.out.print(\"\"enter the number of sides of the polygon ==&gt; \"\");.    numsides = keyboard.nextint();.    system.out.println();..    system.out.print(\"\"enter the side length of each side ==&gt; \"\");.    sidelength = keyboard.nextdouble();.    system.out.println();..    regularpolygon shape = new regularpolygon(numsides, sidelength);.    shape.menu();.}//end main...for testing it i sent it numsides 4 and sidelength 100.\"\\n'\n",
      "Label: 1\n",
      "Question:  b'\"blank code slow skin detection this code changes the color space to lab and using a threshold finds the skin area of an image. but it\\'s ridiculously slow. i don\\'t know how to make it faster ?    ..from colormath.color_objects import *..def skindetection(img, treshold=80, color=[255,20,147]):..    print img.shape.    res=img.copy().    for x in range(img.shape[0]):.        for y in range(img.shape[1]):.            rgbimg=rgbcolor(img[x,y,0],img[x,y,1],img[x,y,2]).            labimg=rgbimg.convert_to(\\'lab\\', debug=false).            if (labimg.lab_l &gt; treshold):.                res[x,y,:]=color.            else: .                res[x,y,:]=img[x,y,:]..    return res\"\\n'\n",
      "Label: 3\n",
      "Question:  b'\"option and validation in blank i want to add a new option on my system where i want to add two text files, both rental.txt and customer.txt. inside each text are id numbers of the customer, the videotape they need and the price...i want to place it as an option on my code. right now i have:...add customer.rent return.view list.search.exit...i want to add this as my sixth option. say for example i ordered a video, it would display the price and would let me confirm the price and if i am going to buy it or not...here is my current code:..  import blank.io.*;.    import blank.util.arraylist;.    import static blank.lang.system.out;..    public class rentalsystem{.    static bufferedreader input = new bufferedreader(new inputstreamreader(system.in));.    static file file = new file(\"\"file.txt\"\");.    static arraylist&lt;string&gt; list = new arraylist&lt;string&gt;();.    static int rows;..    public static void main(string[] args) throws exception{.        introduction();.        system.out.print(\"\"nn\"\");.        login();.        system.out.print(\"\"nnnnnnnnnnnnnnnnnnnnnn\"\");.        introduction();.        string repeat;.        do{.            loadfile();.            system.out.print(\"\"nwhat do you want to do?nn\"\");.            system.out.print(\"\"n                    - - - - - - - - - - - - - - - - - - - - - - -\"\");.            system.out.print(\"\"nn                    |     1. add customer    |   2. rent return |n\"\");.            system.out.print(\"\"n                    - - - - - - - - - - - - - - - - - - - - - - -\"\");.            system.out.print(\"\"nn                    |     3. view list       |   4. search      |n\"\");.            system.out.print(\"\"n                    - - - - - - - - - - - - - - - - - - - - - - -\"\");.            system.out.print(\"\"nn                                             |   5. exit        |n\"\");.            system.out.print(\"\"n                                              - - - - - - - - - -\"\");.            system.out.print(\"\"nnchoice:\"\");.            int choice = integer.parseint(input.readline());.            switch(choice){.                case 1:.                    writedata();.                    break;.                case 2:.                    rentdata();.                    break;.                case 3:.                    viewlist();.                    break;.                case 4:.                    search();.                    break;.                case 5:.                    system.out.println(\"\"goodbye!\"\");.                    system.exit(0);.                default:.                    system.out.print(\"\"invalid choice: \"\");.                    break;.            }.            system.out.print(\"\"ndo another task? [y/n] \"\");.            repeat = input.readline();.        }while(repeat.equals(\"\"y\"\"));..        if(repeat!=\"\"y\"\") system.out.println(\"\"ngoodbye!\"\");..    }..    public static void writedata() throws exception{.        system.out.print(\"\"nname: \"\");.        string cname = input.readline();.        system.out.print(\"\"address: \"\");.        string add = input.readline();.        system.out.print(\"\"phone no.: \"\");.        string pno = input.readline();.        system.out.print(\"\"rental amount: \"\");.        string ramount = input.readline();.        system.out.print(\"\"tapenumber: \"\");.        string tno = input.readline();.        system.out.print(\"\"title: \"\");.        string title = input.readline();.        system.out.print(\"\"date borrowed: \"\");.        string dborrowed = input.readline();.        system.out.print(\"\"due date: \"\");.        string ddate = input.readline();.        createline(cname, add, pno, ramount,tno, title, dborrowed, ddate);.        rentdata();.    }..    public static void createline(string name, string address, string phone , string rental, string tapenumber, string title, string borrowed, string due) throws exception{.        filewriter fw = new filewriter(file, true);.        fw.write(\"\"nname: \"\"+name + \"\"naddress: \"\" + address +\"\"nphone no.: \"\"+ phone+\"\"nrentalamount: \"\"+rental+\"\"ntape no.: \"\"+ tapenumber+\"\"ntitle: \"\"+ title+\"\"ndate borrowed: \"\"+borrowed +\"\"ndue date: \"\"+ due+\"\":rn\"\");.        fw.close();.    }..    public static void loadfile() throws exception{.        try{.            list.clear();.            fileinputstream fstream = new fileinputstream(file);.            bufferedreader br = new bufferedreader(new inputstreamreader(fstream));.            rows = 0;.            while( br.ready()).            {.                list.add(br.readline());.                rows++;.            }.            br.close();.        } catch(exception e){.            system.out.println(\"\"list not yet loaded.\"\");.        }.    }..    public static void viewlist(){.        system.out.print(\"\"n~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\"\");.        system.out.print(\"\" |list of all costumers|\"\");.        system.out.print(\"\"~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\"\");.        for(int i = 0; i &lt;rows; i++){.            system.out.println(list.get(i));.        }.    }.        public static void rentdata()throws exception.    {   system.out.print(\"\"n~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\"\");.        system.out.print(\"\" |rent data list|\"\");.        system.out.print(\"\"~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\"\");.        system.out.print(\"\"nenter customer name: \"\");.        string cname = input.readline();.        system.out.print(\"\"date borrowed: \"\");.        string dborrowed = input.readline();.        system.out.print(\"\"due date: \"\");.        string ddate = input.readline();.        system.out.print(\"\"return date: \"\");.        string rdate = input.readline();.        system.out.print(\"\"rent amount: \"\");.        string ramount = input.readline();..        system.out.print(\"\"you pay:\"\"+ramount);...    }.    public static void search()throws exception.    {   system.out.print(\"\"n~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\"\");.        system.out.print(\"\" |search costumers|\"\");.        system.out.print(\"\"~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~\"\");.        system.out.print(\"\"nenter costumer name: \"\");.        string cname = input.readline();.        boolean found = false;..        for(int i=0; i &lt; rows; i++){.            string temp[] = list.get(i).split(\"\",\"\");..            if(cname.equals(temp[0])){.            system.out.println(\"\"search result:nyou are \"\" + temp[0] + \"\" from \"\" + temp[1] + \"\".\"\"+ temp[2] + \"\".\"\"+ temp[3] + \"\".\"\"+ temp[4] + \"\".\"\"+ temp[5] + \"\" is \"\" + temp[6] + \"\".\"\"+ temp[7] + \"\" is \"\" + temp[8] + \"\".\"\");.                found = true;.            }.        }..        if(!found){.            system.out.print(\"\"no results.\"\");.        }..    }..        public static boolean evaluate(string uname, string pass){.        if (uname.equals(\"\"admin\"\")&amp;&amp;pass.equals(\"\"12345\"\")) return true;.        else return false;.    }..    public static string login()throws exception{.        bufferedreader input=new bufferedreader(new inputstreamreader(system.in));.        int counter=0;.        do{.            system.out.print(\"\"username:\"\");.            string uname =input.readline();.            system.out.print(\"\"password:\"\");.            string pass =input.readline();..            boolean accept= evaluate(uname,pass);..            if(accept){.                break;.                }else{.                    system.out.println(\"\"incorrect username or password!\"\");.                    counter ++;.                    }.        }while(counter&lt;3);..            if(counter !=3) return \"\"login successful\"\";.            else return \"\"login failed\"\";.            }.        public static void introduction() throws exception{..        system.out.println(\"\"                  - - - - - - - - - - - - - - - - - - - - - - - - -\"\");.        system.out.println(\"\"                  !                  r e n t a l                  !\"\");.        system.out.println(\"\"                   ! ~ ~ ~ ~ ~ !  =================  ! ~ ~ ~ ~ ~ !\"\");.        system.out.println(\"\"                  !                  s y s t e m                  !\"\");.        system.out.println(\"\"                  - - - - - - - - - - - - - - - - - - - - - - - - -\"\");.        }..}\"\\n'\n",
      "Label: 1\n",
      "Question:  b'\"exception: dynamic sql generation for the updatecommand is not supported against a selectcommand that does not return any key i dont know what is the problem this my code : ..string nomtable;..datatable listeetablissementtable = new datatable();.datatable listeinteretstable = new datatable();.dataset ds = new dataset();.sqldataadapter da;.sqlcommandbuilder cmdb;..private void listeinterets_click(object sender, eventargs e).{.    nomtable = \"\"listeinteretstable\"\";.    d.cnx.open();.    da = new sqldataadapter(\"\"select nome from offices\"\", d.cnx);.    ds = new dataset();.    da.fill(ds, nomtable);.    datagridview1.datasource = ds.tables[nomtable];.}..private void sauvgarder_click(object sender, eventargs e).{.    d.cnx.open();.    cmdb = new sqlcommandbuilder(da);.    da.update(ds, nomtable);.    d.cnx.close();.}\"\\n'\n",
      "Label: 0\n",
      "Question:  b'\"parameter with question mark and super in blank, i\\'ve come across a method that is formatted like this:..public final subscription subscribe(final action1&lt;? super t&gt; onnext, final action1&lt;throwable&gt; onerror) {.}...in the first parameter, what does the question mark and super mean?\"\\n'\n",
      "Label: 1\n",
      "Question:  b'call two objects wsdl the first time i got a very strange wsdl. ..i would like to call the object (interface - invoicecheck_out) do you know how?....i would like to call the object (variable) do you know how?..try to call (it`s ok)....try to call (how call this?)\\n'\n",
      "Label: 0\n",
      "Question:  b\"how to correctly make the icon for systemtray in blank using icon sizes of any dimension for systemtray doesn't look good overall. .what is the correct way of making icons for windows system tray?..screenshots: http://imgur.com/zsibwn9..icon: http://imgur.com/vsh4zo8\\n\"\n",
      "Label: 0\n",
      "Question:  b'\"is there a way to check a variable that exists in a different script than the original one? i\\'m trying to check if a variable, which was previously set to true in 2.py in 1.py, as 1.py is only supposed to continue if the variable is true...2.py..import os..completed = false..#some stuff here..completed = true...1.py..import 2 ..if completed == true.   #do things...however i get a syntax error at ..if completed == true\"\\n'\n",
      "Label: 3\n",
      "Question:  b'\"blank control flow i made a number which asks for 2 numbers with blank and responds with  the corresponding message for the case. how come it doesnt work  for the second number ? .regardless what i enter for the second number , i am getting the message \"\"your number is in the range 0-10\"\"...using system;.using system.collections.generic;.using system.linq;.using system.text;..namespace consoleapplication1.{.    class program.    {.        static void main(string[] args).        {.            string myinput;  // declaring the type of the variables.            int myint;..            string number1;.            int number;...            console.writeline(\"\"enter a number\"\");.            myinput = console.readline(); //muyinput is a string  which is entry input.            myint = int32.parse(myinput); // myint converts the string into an integer..            if (myint &gt; 0).                console.writeline(\"\"your number {0} is greater than zero.\"\", myint);.            else if (myint &lt; 0).                console.writeline(\"\"your number {0} is  less  than zero.\"\", myint);.            else.                console.writeline(\"\"your number {0} is equal zero.\"\", myint);..            console.writeline(\"\"enter another number\"\");.            number1 = console.readline(); .            number = int32.parse(myinput); ..            if (number &lt; 0 || number == 0).                console.writeline(\"\"your number {0} is  less  than zero or equal zero.\"\", number);.            else if (number &gt; 0 &amp;&amp; number &lt;= 10).                console.writeline(\"\"your number {0} is  in the range from 0 to 10.\"\", number);.            else.                console.writeline(\"\"your number {0} is greater than 10.\"\", number);..            console.writeline(\"\"enter another number\"\");..        }.    }    .}\"\\n'\n",
      "Label: 0\n",
      "Question:  b'\"credentials cannot be used for ntlm authentication i am getting org.apache.commons.httpclient.auth.invalidcredentialsexception: credentials cannot be used for ntlm authentication: exception in eclipse..whether it is possible mention eclipse to take system proxy settings directly?..public class httpgetproxy {.    private static final string proxy_host = \"\"proxy.****.com\"\";.    private static final int proxy_port = 6050;..    public static void main(string[] args) {.        httpclient client = new httpclient();.        httpmethod method = new getmethod(\"\"https://kodeblank.org\"\");..        hostconfiguration config = client.gethostconfiguration();.        config.setproxy(proxy_host, proxy_port);..        string username = \"\"*****\"\";.        string password = \"\"*****\"\";.        credentials credentials = new usernamepasswordcredentials(username, password);.        authscope authscope = new authscope(proxy_host, proxy_port);..        client.getstate().setproxycredentials(authscope, credentials);..        try {.            client.executemethod(method);..            if (method.getstatuscode() == httpstatus.sc_ok) {.                string response = method.getresponsebodyasstring();.                system.out.println(\"\"response = \"\" + response);.            }.        } catch (ioexception e) {.            e.printstacktrace();.        } finally {.            method.releaseconnection();.        }.    }.}...exception:...  dec 08, 2017 1:41:39 pm .          org.apache.commons.httpclient.auth.authchallengeprocessor selectauthscheme.         info: ntlm authentication scheme selected.       dec 08, 2017 1:41:39 pm org.apache.commons.httpclient.httpmethoddirector executeconnect.         severe: credentials cannot be used for ntlm authentication: .           org.apache.commons.httpclient.usernamepasswordcredentials.           org.apache.commons.httpclient.auth.invalidcredentialsexception: credentials .         cannot be used for ntlm authentication: .        enter code here .          org.apache.commons.httpclient.usernamepasswordcredentials.      at org.apache.commons.httpclient.auth.ntlmscheme.authenticate(ntlmscheme.blank:332).        at org.apache.commons.httpclient.httpmethoddirector.authenticateproxy(httpmethoddirector.blank:320).      at org.apache.commons.httpclient.httpmethoddirector.executeconnect(httpmethoddirector.blank:491).      at org.apache.commons.httpclient.httpmethoddirector.executewithretry(httpmethoddirector.blank:391).      at org.apache.commons.httpclient.httpmethoddirector.executemethod(httpmethoddirector.blank:171).      at org.apache.commons.httpclient.httpclient.executemethod(httpclient.blank:397).      at org.apache.commons.httpclient.httpclient.executemethod(httpclient.blank:323).      at httpgetproxy.main(httpgetproxy.blank:31).  dec 08, 2017 1:41:39 pm org.apache.commons.httpclient.httpmethoddirector processproxyauthchallenge.  info: failure authenticating with ntlm @proxy.****.com:6050\"\\n'\n",
      "Label: 1\n"
     ]
    }
   ],
   "source": [
    "for text_batch, label_batch in raw_train_ds.take(1):\n",
    "  for i in range(10):\n",
    "    print(\"Question: \", text_batch.numpy()[i])\n",
    "    print(\"Label:\", label_batch.numpy()[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jCZGl4Q5l2sS"
   },
   "source": [
    "标签为 `0`、`1`、`2` 或 `3`。要查看其中哪些对应于哪个字符串标签，可以检查数据集上的 `class_names` 属性：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:55.917851Z",
     "iopub.status.busy": "2022-08-31T05:39:55.917225Z",
     "iopub.status.idle": "2022-08-31T05:39:55.921179Z",
     "shell.execute_reply": "2022-08-31T05:39:55.920539Z"
    },
    "id": "gIpCS7YjmGkj"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label 0 corresponds to csharp\n",
      "Label 1 corresponds to java\n",
      "Label 2 corresponds to javascript\n",
      "Label 3 corresponds to python\n"
     ]
    }
   ],
   "source": [
    "for i, label in enumerate(raw_train_ds.class_names):\n",
    "  print(\"Label\", i, \"corresponds to\", label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oUsdn-37qol9"
   },
   "source": [
    "接下来，您将使用 `tf.keras.utils.text_dataset_from_directory` 创建验证集和测试集。您将使用训练集中剩余的 1,600 条评论进行验证。\n",
    "\n",
    "注：使用 `tf.keras.utils.text_dataset_from_directory` 的 `validation_split` 和 `subset` 参数时，请确保要么指定随机种子，要么传递 `shuffle=False`，这样验证拆分和训练拆分就不会重叠。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:55.924771Z",
     "iopub.status.busy": "2022-08-31T05:39:55.924281Z",
     "iopub.status.idle": "2022-08-31T05:39:56.105808Z",
     "shell.execute_reply": "2022-08-31T05:39:56.105091Z"
    },
    "id": "x7m6sCWJQuYt"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 8000 files belonging to 4 classes.\n",
      "Using 1600 files for validation.\n"
     ]
    }
   ],
   "source": [
    "# Create a validation set.\n",
    "raw_val_ds = utils.text_dataset_from_directory(\n",
    "    train_dir,\n",
    "    batch_size=batch_size,\n",
    "    validation_split=0.2,\n",
    "    subset='validation',\n",
    "    seed=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:56.109706Z",
     "iopub.status.busy": "2022-08-31T05:39:56.109221Z",
     "iopub.status.idle": "2022-08-31T05:39:56.295115Z",
     "shell.execute_reply": "2022-08-31T05:39:56.294455Z"
    },
    "id": "BXMZc7fMQwKE"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found 8000 files belonging to 4 classes.\n"
     ]
    }
   ],
   "source": [
    "test_dir = dataset_dir/'test'\n",
    "\n",
    "# Create a test set.\n",
    "raw_test_ds = utils.text_dataset_from_directory(\n",
    "    test_dir,\n",
    "    batch_size=batch_size)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Xdt-ATrGRGDL"
   },
   "source": [
    "### 准备用于训练的数据集"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N6fRti45Rlj8"
   },
   "source": [
    "接下来，您将使用 `tf.keras.layers.TextVectorization` 层对数据进行标准化、词例化和向量化。\n",
    "\n",
    "- *标准化*是指预处理文本，通常是移除标点符号或 HTML 元素以简化数据集。\n",
    "- *词例化*是指将字符串拆分为词例（例如，通过按空格分割将一个句子拆分为各个单词）。\n",
    "- *向量化*是指将词例转换为编号，以便将它们输入到神经网络中。\n",
    "\n",
    "所有这些任务都可以通过这一层来完成。（您可以在 `tf.keras.layers.TextVectorization` API 文档中了解有关这些内容的更多信息。）\n",
    "\n",
    "请注意：\n",
    "\n",
    "- 默认标准化会将文本转换为小写并移除标点符号 (`standardize='lower_and_strip_punctuation'`)。\n",
    "- 默认分词器会按空格分割 (`split='whitespace'`)。\n",
    "- 默认向量化模式为 `'int'` (`output_mode='int'`)。这会输出整数索引（每个词例一个）。此模式可用于构建考虑词序的模型。您还可以使用其他模式（例如 `'binary'`）来构建[词袋](https://developers.google.com/machine-learning/glossary#bag-of-words)模型。\n",
    "\n",
    "您将使用 `TextVectorization` 构建两个模型来详细了解标准化、词例化和向量化：\n",
    "\n",
    "- 首先，您将使用 `'binary'` 向量化模式来构建词袋模型。\n",
    "- 随后，您将使用具有 1D ConvNet 的 `'int'` 模式。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:56.299810Z",
     "iopub.status.busy": "2022-08-31T05:39:56.299271Z",
     "iopub.status.idle": "2022-08-31T05:39:56.311239Z",
     "shell.execute_reply": "2022-08-31T05:39:56.310649Z"
    },
    "id": "voaC43rZR0jc"
   },
   "outputs": [],
   "source": [
    "VOCAB_SIZE = 10000\n",
    "\n",
    "binary_vectorize_layer = TextVectorization(\n",
    "    max_tokens=VOCAB_SIZE,\n",
    "    output_mode='binary')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ifDPFxuf2Hfz"
   },
   "source": [
    "对于 `'int'` 模式，除了最大词汇量之外，您还需要设置显式最大序列长度 (`MAX_SEQUENCE_LENGTH`)，这会导致层将序列精确地填充或截断为 `output_sequence_length` 值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:56.315177Z",
     "iopub.status.busy": "2022-08-31T05:39:56.314621Z",
     "iopub.status.idle": "2022-08-31T05:39:56.320576Z",
     "shell.execute_reply": "2022-08-31T05:39:56.319985Z"
    },
    "id": "XWsY01Zl2aRe"
   },
   "outputs": [],
   "source": [
    "MAX_SEQUENCE_LENGTH = 250\n",
    "\n",
    "int_vectorize_layer = TextVectorization(\n",
    "    max_tokens=VOCAB_SIZE,\n",
    "    output_mode='int',\n",
    "    output_sequence_length=MAX_SEQUENCE_LENGTH)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ts6h9b5atD-Y"
   },
   "source": [
    "接下来，调用 `TextVectorization.adapt` 以使预处理层的状态适合数据集。这会使模型构建字符串到整数的索引。\n",
    "\n",
    "注：在调用 `TextVectorization.adapt` 时请务必仅使用您的训练数据（使用测试集会泄漏信息）。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:56.323728Z",
     "iopub.status.busy": "2022-08-31T05:39:56.323471Z",
     "iopub.status.idle": "2022-08-31T05:39:58.074938Z",
     "shell.execute_reply": "2022-08-31T05:39:58.074183Z"
    },
    "id": "yTXsdDEqSf9e"
   },
   "outputs": [],
   "source": [
    "# Make a text-only dataset (without labels), then call `TextVectorization.adapt`.\n",
    "train_text = raw_train_ds.map(lambda text, labels: text)\n",
    "binary_vectorize_layer.adapt(train_text)\n",
    "int_vectorize_layer.adapt(train_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XKVO6Jg7Sls0"
   },
   "source": [
    "打印使用这些层预处理数据的结果："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.079866Z",
     "iopub.status.busy": "2022-08-31T05:39:58.079341Z",
     "iopub.status.idle": "2022-08-31T05:39:58.083297Z",
     "shell.execute_reply": "2022-08-31T05:39:58.082696Z"
    },
    "id": "RngfPyArSsvM"
   },
   "outputs": [],
   "source": [
    "def binary_vectorize_text(text, label):\n",
    "  text = tf.expand_dims(text, -1)\n",
    "  return binary_vectorize_layer(text), label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.086745Z",
     "iopub.status.busy": "2022-08-31T05:39:58.086154Z",
     "iopub.status.idle": "2022-08-31T05:39:58.089648Z",
     "shell.execute_reply": "2022-08-31T05:39:58.089097Z"
    },
    "id": "_1W54wf0LhQ0"
   },
   "outputs": [],
   "source": [
    "def int_vectorize_text(text, label):\n",
    "  text = tf.expand_dims(text, -1)\n",
    "  return int_vectorize_layer(text), label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.092691Z",
     "iopub.status.busy": "2022-08-31T05:39:58.092213Z",
     "iopub.status.idle": "2022-08-31T05:39:58.112664Z",
     "shell.execute_reply": "2022-08-31T05:39:58.111949Z"
    },
    "id": "Vi_sElMiSmXe"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Question tf.Tensor(b'\"what is the difference between these two ways to create an element? var a = document.createelement(\\'div\\');..a.id = \"\"mydiv\"\";...and..var a = document.createelement(\\'div\\').id = \"\"mydiv\"\";...what is the difference between them such that the first one works and the second one doesn\\'t?\"\\n', shape=(), dtype=string)\n",
      "Label tf.Tensor(2, shape=(), dtype=int32)\n"
     ]
    }
   ],
   "source": [
    "# Retrieve a batch (of 32 reviews and labels) from the dataset.\n",
    "text_batch, label_batch = next(iter(raw_train_ds))\n",
    "first_question, first_label = text_batch[0], label_batch[0]\n",
    "print(\"Question\", first_question)\n",
    "print(\"Label\", first_label)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.116308Z",
     "iopub.status.busy": "2022-08-31T05:39:58.115643Z",
     "iopub.status.idle": "2022-08-31T05:39:58.130345Z",
     "shell.execute_reply": "2022-08-31T05:39:58.129762Z"
    },
    "id": "UGukZoYv2v3v"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'binary' vectorized question: tf.Tensor([[1. 1. 0. ... 0. 0. 0.]], shape=(1, 10000), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "print(\"'binary' vectorized question:\",\n",
    "      binary_vectorize_text(first_question, first_label)[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.133690Z",
     "iopub.status.busy": "2022-08-31T05:39:58.133175Z",
     "iopub.status.idle": "2022-08-31T05:39:58.142450Z",
     "shell.execute_reply": "2022-08-31T05:39:58.141785Z"
    },
    "id": "Lu07FsIw2yH5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'int' vectorized question: tf.Tensor(\n",
      "[[ 55   6   2 410 211 229 121 895   4 124  32 245  43   5   1   1   5   1\n",
      "    1   6   2 410 211 191 318  14   2  98  71 188   8   2 199  71 178   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0\n",
      "    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0]], shape=(1, 250), dtype=int64)\n"
     ]
    }
   ],
   "source": [
    "print(\"'int' vectorized question:\",\n",
    "      int_vectorize_text(first_question, first_label)[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wgjeF9PdS7tN"
   },
   "source": [
    "如上所示，`TextVectorization` 的 `'binary'` 模式返回一个数组，表示哪些词例在输入中至少存在一次，而 `'int'` 模式将每个词例替换为一个整数，从而保留它们的顺序。\n",
    "\n",
    "您可以通过在层上调用 `TextVectorization.get_vocabulary` 来查找每个整数对应的词例（字符串）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.146129Z",
     "iopub.status.busy": "2022-08-31T05:39:58.145719Z",
     "iopub.status.idle": "2022-08-31T05:39:58.191186Z",
     "shell.execute_reply": "2022-08-31T05:39:58.190553Z"
    },
    "id": "WpBnTZilS8wt"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1289 --->  roman\n",
      "313 --->  source\n",
      "Vocabulary size: 10000\n"
     ]
    }
   ],
   "source": [
    "print(\"1289 ---> \", int_vectorize_layer.get_vocabulary()[1289])\n",
    "print(\"313 ---> \", int_vectorize_layer.get_vocabulary()[313])\n",
    "print(\"Vocabulary size: {}\".format(len(int_vectorize_layer.get_vocabulary())))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0kHgPE_YwHvp"
   },
   "source": [
    "差不多可以训练您的模型了。\n",
    "\n",
    "作为最后的预处理步骤，将之前创建的 `TextVectorization` 层应用于训练集、验证集和测试集："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.194871Z",
     "iopub.status.busy": "2022-08-31T05:39:58.194362Z",
     "iopub.status.idle": "2022-08-31T05:39:58.439940Z",
     "shell.execute_reply": "2022-08-31T05:39:58.439260Z"
    },
    "id": "46LeHmnD55wJ"
   },
   "outputs": [],
   "source": [
    "binary_train_ds = raw_train_ds.map(binary_vectorize_text)\n",
    "binary_val_ds = raw_val_ds.map(binary_vectorize_text)\n",
    "binary_test_ds = raw_test_ds.map(binary_vectorize_text)\n",
    "\n",
    "int_train_ds = raw_train_ds.map(int_vectorize_text)\n",
    "int_val_ds = raw_val_ds.map(int_vectorize_text)\n",
    "int_test_ds = raw_test_ds.map(int_vectorize_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NHuAF8hYfP5Z"
   },
   "source": [
    "### 配置数据集以提高性能\n",
    "\n",
    "以下是加载数据时应该使用的两种重要方法，以确保 I/O 不会阻塞。\n",
    "\n",
    "- 从磁盘加载后，`Dataset.cache` 会将数据保存在内存中。这将确保数据集在训练模型时不会成为瓶颈。如果您的数据集太大而无法放入内存，也可以使用此方法创建高性能的磁盘缓存，这比许多小文件的读取效率更高。\n",
    "- `Dataset.prefetch` 会在训练时将数据预处理和模型执行重叠。\n",
    "\n",
    "您可以在[使用 tf.data API 提升性能](../../guide/data_performance.ipynb)指南的*预提取*部分中详细了解这两种方法，以及如何将数据缓存到磁盘。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.444277Z",
     "iopub.status.busy": "2022-08-31T05:39:58.444004Z",
     "iopub.status.idle": "2022-08-31T05:39:58.447847Z",
     "shell.execute_reply": "2022-08-31T05:39:58.447168Z"
    },
    "id": "PabA9DFIfSz7"
   },
   "outputs": [],
   "source": [
    "AUTOTUNE = tf.data.AUTOTUNE\n",
    "\n",
    "def configure_dataset(dataset):\n",
    "  return dataset.cache().prefetch(buffer_size=AUTOTUNE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.451044Z",
     "iopub.status.busy": "2022-08-31T05:39:58.450801Z",
     "iopub.status.idle": "2022-08-31T05:39:58.458942Z",
     "shell.execute_reply": "2022-08-31T05:39:58.458320Z"
    },
    "id": "J8GcJLvb3JH0"
   },
   "outputs": [],
   "source": [
    "binary_train_ds = configure_dataset(binary_train_ds)\n",
    "binary_val_ds = configure_dataset(binary_val_ds)\n",
    "binary_test_ds = configure_dataset(binary_test_ds)\n",
    "\n",
    "int_train_ds = configure_dataset(int_train_ds)\n",
    "int_val_ds = configure_dataset(int_val_ds)\n",
    "int_test_ds = configure_dataset(int_test_ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "NYGb7z_bfpGm"
   },
   "source": [
    "### 训练模型。\n",
    "\n",
    "是时候创建您的神经网络了。\n",
    "\n",
    "对于 `'binary'` 向量化数据，定义一个简单的词袋线性模型，然后对其进行配置和训练："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:39:58.462771Z",
     "iopub.status.busy": "2022-08-31T05:39:58.462521Z",
     "iopub.status.idle": "2022-08-31T05:40:05.260629Z",
     "shell.execute_reply": "2022-08-31T05:40:05.259848Z"
    },
    "id": "2q8iAU-VMzaN"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 1:54 - loss: 1.3697 - accuracy: 0.2812"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 18/200 [=>............................] - ETA: 0s - loss: 1.3616 - accuracy: 0.3056  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 37/200 [====>.........................] - ETA: 0s - loss: 1.3399 - accuracy: 0.3826"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 56/200 [=======>......................] - ETA: 0s - loss: 1.3046 - accuracy: 0.4682"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 74/200 [==========>...................] - ETA: 0s - loss: 1.2756 - accuracy: 0.5156"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 94/200 [=============>................] - ETA: 0s - loss: 1.2471 - accuracy: 0.5552"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "113/200 [===============>..............] - ETA: 0s - loss: 1.2195 - accuracy: 0.5860"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "132/200 [==================>...........] - ETA: 0s - loss: 1.1976 - accuracy: 0.6039"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "152/200 [=====================>........] - ETA: 0s - loss: 1.1730 - accuracy: 0.6209"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "171/200 [========================>.....] - ETA: 0s - loss: 1.1496 - accuracy: 0.6308"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "189/200 [===========================>..] - ETA: 0s - loss: 1.1323 - accuracy: 0.6400"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 4ms/step - loss: 1.1207 - accuracy: 0.6450 - val_loss: 0.9159 - val_accuracy: 0.7750\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 2/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.8758 - accuracy: 0.8125"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 25/200 [==>...........................] - ETA: 0s - loss: 0.8584 - accuracy: 0.8150"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 49/200 [======>.......................] - ETA: 0s - loss: 0.8523 - accuracy: 0.8151"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 72/200 [=========>....................] - ETA: 0s - loss: 0.8342 - accuracy: 0.8186"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 95/200 [=============>................] - ETA: 0s - loss: 0.8254 - accuracy: 0.8151"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "118/200 [================>.............] - ETA: 0s - loss: 0.8127 - accuracy: 0.8181"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "141/200 [====================>.........] - ETA: 0s - loss: 0.8062 - accuracy: 0.8160"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "163/200 [=======================>......] - ETA: 0s - loss: 0.7938 - accuracy: 0.8146"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "185/200 [==========================>...] - ETA: 0s - loss: 0.7852 - accuracy: 0.8166"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.7795 - accuracy: 0.8163 - val_loss: 0.7511 - val_accuracy: 0.8006\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 3/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.7493 - accuracy: 0.7500"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 24/200 [==>...........................] - ETA: 0s - loss: 0.6772 - accuracy: 0.8555"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 47/200 [======>.......................] - ETA: 0s - loss: 0.6757 - accuracy: 0.8577"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 70/200 [=========>....................] - ETA: 0s - loss: 0.6574 - accuracy: 0.8621"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 93/200 [============>.................] - ETA: 0s - loss: 0.6569 - accuracy: 0.8589"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "116/200 [================>.............] - ETA: 0s - loss: 0.6465 - accuracy: 0.8602"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "139/200 [===================>..........] - ETA: 0s - loss: 0.6454 - accuracy: 0.8604"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "164/200 [=======================>......] - ETA: 0s - loss: 0.6361 - accuracy: 0.8603"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "188/200 [===========================>..] - ETA: 0s - loss: 0.6302 - accuracy: 0.8619"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.6278 - accuracy: 0.8619 - val_loss: 0.6652 - val_accuracy: 0.8169\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 4/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.6738 - accuracy: 0.8438"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 24/200 [==>...........................] - ETA: 0s - loss: 0.5723 - accuracy: 0.8854"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 47/200 [======>.......................] - ETA: 0s - loss: 0.5708 - accuracy: 0.8890"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 70/200 [=========>....................] - ETA: 0s - loss: 0.5542 - accuracy: 0.8911"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 93/200 [============>.................] - ETA: 0s - loss: 0.5555 - accuracy: 0.8844"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "116/200 [================>.............] - ETA: 0s - loss: 0.5467 - accuracy: 0.8844"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "139/200 [===================>..........] - ETA: 0s - loss: 0.5470 - accuracy: 0.8847"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "163/200 [=======================>......] - ETA: 0s - loss: 0.5401 - accuracy: 0.8863"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "187/200 [===========================>..] - ETA: 0s - loss: 0.5358 - accuracy: 0.8870"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.5341 - accuracy: 0.8869 - val_loss: 0.6117 - val_accuracy: 0.8294\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 5/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.6173 - accuracy: 0.8750"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 25/200 [==>...........................] - ETA: 0s - loss: 0.4993 - accuracy: 0.9038"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 50/200 [======>.......................] - ETA: 0s - loss: 0.4939 - accuracy: 0.9044"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 74/200 [==========>...................] - ETA: 0s - loss: 0.4842 - accuracy: 0.9050"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 98/200 [=============>................] - ETA: 0s - loss: 0.4828 - accuracy: 0.9011"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "122/200 [=================>............] - ETA: 0s - loss: 0.4797 - accuracy: 0.8996"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "146/200 [====================>.........] - ETA: 0s - loss: 0.4776 - accuracy: 0.9005"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "171/200 [========================>.....] - ETA: 0s - loss: 0.4716 - accuracy: 0.9022"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "195/200 [============================>.] - ETA: 0s - loss: 0.4693 - accuracy: 0.9040"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.4680 - accuracy: 0.9041 - val_loss: 0.5751 - val_accuracy: 0.8369\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 6/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.5713 - accuracy: 0.8750"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 25/200 [==>...........................] - ETA: 0s - loss: 0.4455 - accuracy: 0.9162"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 49/200 [======>.......................] - ETA: 0s - loss: 0.4422 - accuracy: 0.9177"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 74/200 [==========>...................] - ETA: 0s - loss: 0.4311 - accuracy: 0.9189"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 99/200 [=============>................] - ETA: 0s - loss: 0.4296 - accuracy: 0.9148"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "123/200 [=================>............] - ETA: 0s - loss: 0.4279 - accuracy: 0.9131"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "146/200 [====================>.........] - ETA: 0s - loss: 0.4258 - accuracy: 0.9142"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "169/200 [========================>.....] - ETA: 0s - loss: 0.4212 - accuracy: 0.9159"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "193/200 [===========================>..] - ETA: 0s - loss: 0.4193 - accuracy: 0.9181"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.4177 - accuracy: 0.9181 - val_loss: 0.5485 - val_accuracy: 0.8413\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 7/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.5321 - accuracy: 0.8750"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 24/200 [==>...........................] - ETA: 0s - loss: 0.4042 - accuracy: 0.9232"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 46/200 [=====>........................] - ETA: 0s - loss: 0.3991 - accuracy: 0.9280"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 69/200 [=========>....................] - ETA: 0s - loss: 0.3881 - accuracy: 0.9298"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 92/200 [============>.................] - ETA: 0s - loss: 0.3901 - accuracy: 0.9246"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "115/200 [================>.............] - ETA: 0s - loss: 0.3840 - accuracy: 0.9255"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "138/200 [===================>..........] - ETA: 0s - loss: 0.3844 - accuracy: 0.9257"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "160/200 [=======================>......] - ETA: 0s - loss: 0.3804 - accuracy: 0.9279"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "183/200 [==========================>...] - ETA: 0s - loss: 0.3790 - accuracy: 0.9281"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.3775 - accuracy: 0.9292 - val_loss: 0.5284 - val_accuracy: 0.8400\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 8/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.4979 - accuracy: 0.8750"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 23/200 [==>...........................] - ETA: 0s - loss: 0.3657 - accuracy: 0.9321"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 46/200 [=====>........................] - ETA: 0s - loss: 0.3637 - accuracy: 0.9355"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 68/200 [=========>....................] - ETA: 0s - loss: 0.3538 - accuracy: 0.9366"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 91/200 [============>.................] - ETA: 0s - loss: 0.3560 - accuracy: 0.9330"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "116/200 [================>.............] - ETA: 0s - loss: 0.3498 - accuracy: 0.9343"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "141/200 [====================>.........] - ETA: 0s - loss: 0.3502 - accuracy: 0.9346"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "164/200 [=======================>......] - ETA: 0s - loss: 0.3471 - accuracy: 0.9358"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "188/200 [===========================>..] - ETA: 0s - loss: 0.3451 - accuracy: 0.9372"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.3442 - accuracy: 0.9373 - val_loss: 0.5129 - val_accuracy: 0.8425\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 9/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.4676 - accuracy: 0.8750"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 23/200 [==>...........................] - ETA: 0s - loss: 0.3363 - accuracy: 0.9334"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 45/200 [=====>........................] - ETA: 0s - loss: 0.3333 - accuracy: 0.9375"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 67/200 [=========>....................] - ETA: 0s - loss: 0.3248 - accuracy: 0.9394"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 90/200 [============>.................] - ETA: 0s - loss: 0.3277 - accuracy: 0.9375"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "112/200 [===============>..............] - ETA: 0s - loss: 0.3222 - accuracy: 0.9395"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "135/200 [===================>..........] - ETA: 0s - loss: 0.3213 - accuracy: 0.9398"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "158/200 [======================>.......] - ETA: 0s - loss: 0.3186 - accuracy: 0.9409"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "181/200 [==========================>...] - ETA: 0s - loss: 0.3167 - accuracy: 0.9423"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.3160 - accuracy: 0.9425 - val_loss: 0.5007 - val_accuracy: 0.8444\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 10/10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.4404 - accuracy: 0.8750"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 23/200 [==>...........................] - ETA: 0s - loss: 0.3111 - accuracy: 0.9402"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 45/200 [=====>........................] - ETA: 0s - loss: 0.3077 - accuracy: 0.9451"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 68/200 [=========>....................] - ETA: 0s - loss: 0.3001 - accuracy: 0.9472"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 90/200 [============>.................] - ETA: 0s - loss: 0.3026 - accuracy: 0.9448"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "112/200 [===============>..............] - ETA: 0s - loss: 0.2975 - accuracy: 0.9461"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "134/200 [===================>..........] - ETA: 0s - loss: 0.2963 - accuracy: 0.9475"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "156/200 [======================>.......] - ETA: 0s - loss: 0.2942 - accuracy: 0.9489"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "179/200 [=========================>....] - ETA: 0s - loss: 0.2924 - accuracy: 0.9497"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 3ms/step - loss: 0.2917 - accuracy: 0.9497 - val_loss: 0.4910 - val_accuracy: 0.8419\n"
     ]
    }
   ],
   "source": [
    "binary_model = tf.keras.Sequential([layers.Dense(4)])\n",
    "\n",
    "binary_model.compile(\n",
    "    loss=losses.SparseCategoricalCrossentropy(from_logits=True),\n",
    "    optimizer='adam',\n",
    "    metrics=['accuracy'])\n",
    "\n",
    "history = binary_model.fit(\n",
    "    binary_train_ds, validation_data=binary_val_ds, epochs=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EwidD-SwNIkz"
   },
   "source": [
    "接下来，您将使用 `'int'` 向量化层来构建 1D ConvNet："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:05.264888Z",
     "iopub.status.busy": "2022-08-31T05:40:05.264618Z",
     "iopub.status.idle": "2022-08-31T05:40:05.269244Z",
     "shell.execute_reply": "2022-08-31T05:40:05.268546Z"
    },
    "id": "5ztw2XH_LbVz"
   },
   "outputs": [],
   "source": [
    "def create_model(vocab_size, num_labels):\n",
    "  model = tf.keras.Sequential([\n",
    "      layers.Embedding(vocab_size, 64, mask_zero=True),\n",
    "      layers.Conv1D(64, 5, padding=\"valid\", activation=\"relu\", strides=2),\n",
    "      layers.GlobalMaxPooling1D(),\n",
    "      layers.Dense(num_labels)\n",
    "  ])\n",
    "  return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:05.272485Z",
     "iopub.status.busy": "2022-08-31T05:40:05.271930Z",
     "iopub.status.idle": "2022-08-31T05:40:10.552395Z",
     "shell.execute_reply": "2022-08-31T05:40:10.551593Z"
    },
    "id": "s9rG1cFRL31Z"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 4:22 - loss: 1.3986 - accuracy: 0.1875"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 16/200 [=>............................] - ETA: 0s - loss: 1.3876 - accuracy: 0.2422  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 32/200 [===>..........................] - ETA: 0s - loss: 1.3828 - accuracy: 0.2617"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 48/200 [======>.......................] - ETA: 0s - loss: 1.3753 - accuracy: 0.3216"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 64/200 [========>.....................] - ETA: 0s - loss: 1.3641 - accuracy: 0.3638"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 80/200 [===========>..................] - ETA: 0s - loss: 1.3462 - accuracy: 0.3953"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 96/200 [=============>................] - ETA: 0s - loss: 1.3199 - accuracy: 0.4007"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "113/200 [===============>..............] - ETA: 0s - loss: 1.2860 - accuracy: 0.4179"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "130/200 [==================>...........] - ETA: 0s - loss: 1.2530 - accuracy: 0.4363"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "146/200 [====================>.........] - ETA: 0s - loss: 1.2252 - accuracy: 0.4495"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "163/200 [=======================>......] - ETA: 0s - loss: 1.1944 - accuracy: 0.4647"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "179/200 [=========================>....] - ETA: 0s - loss: 1.1626 - accuracy: 0.4825"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "194/200 [============================>.] - ETA: 0s - loss: 1.1387 - accuracy: 0.4929"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 2s 5ms/step - loss: 1.1306 - accuracy: 0.4970 - val_loss: 0.7709 - val_accuracy: 0.6919\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 2/5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.9001 - accuracy: 0.6562"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 18/200 [=>............................] - ETA: 0s - loss: 0.8410 - accuracy: 0.6424"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 35/200 [====>.........................] - ETA: 0s - loss: 0.7693 - accuracy: 0.6804"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 52/200 [======>.......................] - ETA: 0s - loss: 0.7469 - accuracy: 0.6947"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 69/200 [=========>....................] - ETA: 0s - loss: 0.7298 - accuracy: 0.6993"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 86/200 [===========>..................] - ETA: 0s - loss: 0.7197 - accuracy: 0.7028"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "103/200 [==============>...............] - ETA: 0s - loss: 0.6993 - accuracy: 0.7166"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "120/200 [=================>............] - ETA: 0s - loss: 0.6848 - accuracy: 0.7258"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "137/200 [===================>..........] - ETA: 0s - loss: 0.6737 - accuracy: 0.7322"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "154/200 [======================>.......] - ETA: 0s - loss: 0.6561 - accuracy: 0.7427"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "171/200 [========================>.....] - ETA: 0s - loss: 0.6438 - accuracy: 0.7496"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "188/200 [===========================>..] - ETA: 0s - loss: 0.6286 - accuracy: 0.7565"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 4ms/step - loss: 0.6226 - accuracy: 0.7592 - val_loss: 0.5376 - val_accuracy: 0.7969\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 3/5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.5307 - accuracy: 0.7812"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 18/200 [=>............................] - ETA: 0s - loss: 0.5370 - accuracy: 0.8073"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 35/200 [====>.........................] - ETA: 0s - loss: 0.4714 - accuracy: 0.8357"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 52/200 [======>.......................] - ETA: 0s - loss: 0.4552 - accuracy: 0.8552"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 69/200 [=========>....................] - ETA: 0s - loss: 0.4429 - accuracy: 0.8555"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 86/200 [===========>..................] - ETA: 0s - loss: 0.4391 - accuracy: 0.8557"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "103/200 [==============>...............] - ETA: 0s - loss: 0.4260 - accuracy: 0.8623"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "120/200 [=================>............] - ETA: 0s - loss: 0.4155 - accuracy: 0.8682"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "137/200 [===================>..........] - ETA: 0s - loss: 0.4067 - accuracy: 0.8707"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "154/200 [======================>.......] - ETA: 0s - loss: 0.3949 - accuracy: 0.8768"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "171/200 [========================>.....] - ETA: 0s - loss: 0.3874 - accuracy: 0.8805"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "188/200 [===========================>..] - ETA: 0s - loss: 0.3775 - accuracy: 0.8850"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 4ms/step - loss: 0.3735 - accuracy: 0.8863 - val_loss: 0.4675 - val_accuracy: 0.8163\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 4/5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.3156 - accuracy: 0.9375"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 18/200 [=>............................] - ETA: 0s - loss: 0.3272 - accuracy: 0.9115"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 35/200 [====>.........................] - ETA: 0s - loss: 0.2775 - accuracy: 0.9196"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 52/200 [======>.......................] - ETA: 0s - loss: 0.2653 - accuracy: 0.9273"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 69/200 [=========>....................] - ETA: 0s - loss: 0.2550 - accuracy: 0.9330"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 86/200 [===========>..................] - ETA: 0s - loss: 0.2504 - accuracy: 0.9346"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "103/200 [==============>...............] - ETA: 0s - loss: 0.2407 - accuracy: 0.9390"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "120/200 [=================>............] - ETA: 0s - loss: 0.2330 - accuracy: 0.9404"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "137/200 [===================>..........] - ETA: 0s - loss: 0.2275 - accuracy: 0.9423"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "154/200 [======================>.......] - ETA: 0s - loss: 0.2207 - accuracy: 0.9456"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "171/200 [========================>.....] - ETA: 0s - loss: 0.2164 - accuracy: 0.9470"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "188/200 [===========================>..] - ETA: 0s - loss: 0.2095 - accuracy: 0.9495"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 4ms/step - loss: 0.2061 - accuracy: 0.9503 - val_loss: 0.4679 - val_accuracy: 0.8163\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 5/5\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/200 [..............................] - ETA: 0s - loss: 0.1453 - accuracy: 1.0000"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 18/200 [=>............................] - ETA: 0s - loss: 0.1744 - accuracy: 0.9601"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 35/200 [====>.........................] - ETA: 0s - loss: 0.1449 - accuracy: 0.9688"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 52/200 [======>.......................] - ETA: 0s - loss: 0.1382 - accuracy: 0.9712"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 69/200 [=========>....................] - ETA: 0s - loss: 0.1307 - accuracy: 0.9737"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 86/200 [===========>..................] - ETA: 0s - loss: 0.1254 - accuracy: 0.9760"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "103/200 [==============>...............] - ETA: 0s - loss: 0.1193 - accuracy: 0.9779"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "120/200 [=================>............] - ETA: 0s - loss: 0.1151 - accuracy: 0.9786"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "137/200 [===================>..........] - ETA: 0s - loss: 0.1126 - accuracy: 0.9790"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "154/200 [======================>.......] - ETA: 0s - loss: 0.1097 - accuracy: 0.9797"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "170/200 [========================>.....] - ETA: 0s - loss: 0.1079 - accuracy: 0.9805"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "187/200 [===========================>..] - ETA: 0s - loss: 0.1038 - accuracy: 0.9815"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/200 [==============================] - 1s 4ms/step - loss: 0.1014 - accuracy: 0.9820 - val_loss: 0.4937 - val_accuracy: 0.8119\n"
     ]
    }
   ],
   "source": [
    "# `vocab_size` is `VOCAB_SIZE + 1` since `0` is used additionally for padding.\n",
    "int_model = create_model(vocab_size=VOCAB_SIZE + 1, num_labels=4)\n",
    "int_model.compile(\n",
    "    loss=losses.SparseCategoricalCrossentropy(from_logits=True),\n",
    "    optimizer='adam',\n",
    "    metrics=['accuracy'])\n",
    "history = int_model.fit(int_train_ds, validation_data=int_val_ds, epochs=5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "x3J9Eeuv97zE"
   },
   "source": [
    "比较两个模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:10.556347Z",
     "iopub.status.busy": "2022-08-31T05:40:10.556057Z",
     "iopub.status.idle": "2022-08-31T05:40:10.566558Z",
     "shell.execute_reply": "2022-08-31T05:40:10.565912Z"
    },
    "id": "N8ViDXw99v_u"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Linear model on binary vectorized data:\n",
      "Model: \"sequential\"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Layer (type)                Output Shape              Param #   \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=================================================================\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " dense (Dense)               (None, 4)                 40004     \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=================================================================\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total params: 40,004\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trainable params: 40,004\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Non-trainable params: 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(\"Linear model on binary vectorized data:\")\n",
    "print(binary_model.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:10.571566Z",
     "iopub.status.busy": "2022-08-31T05:40:10.571345Z",
     "iopub.status.idle": "2022-08-31T05:40:10.584361Z",
     "shell.execute_reply": "2022-08-31T05:40:10.583704Z"
    },
    "id": "P9BOeoCwborD"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ConvNet model on int vectorized data:\n",
      "Model: \"sequential_1\"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Layer (type)                Output Shape              Param #   \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=================================================================\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " embedding (Embedding)       (None, None, 64)          640064    \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " conv1d (Conv1D)             (None, None, 64)          20544     \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " global_max_pooling1d (Globa  (None, 64)               0         \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " lMaxPooling1D)                                                  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " dense_1 (Dense)             (None, 4)                 260       \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=================================================================\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total params: 660,868\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trainable params: 660,868\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Non-trainable params: 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "None\n"
     ]
    }
   ],
   "source": [
    "print(\"ConvNet model on int vectorized data:\")\n",
    "print(int_model.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zYYW9tUdCtTy"
   },
   "source": [
    "在测试数据上评估两个模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:10.592435Z",
     "iopub.status.busy": "2022-08-31T05:40:10.592206Z",
     "iopub.status.idle": "2022-08-31T05:40:11.974709Z",
     "shell.execute_reply": "2022-08-31T05:40:11.973879Z"
    },
    "id": "5dTc4nZqf7fK"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/250 [..............................] - ETA: 20s - loss: 0.4025 - accuracy: 0.9375"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 19/250 [=>............................] - ETA: 0s - loss: 0.5244 - accuracy: 0.8141 "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 38/250 [===>..........................] - ETA: 0s - loss: 0.5201 - accuracy: 0.8191"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 57/250 [=====>........................] - ETA: 0s - loss: 0.5260 - accuracy: 0.8147"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 76/250 [========>.....................] - ETA: 0s - loss: 0.5292 - accuracy: 0.8100"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 94/250 [==========>...................] - ETA: 0s - loss: 0.5228 - accuracy: 0.8142"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "112/250 [============>.................] - ETA: 0s - loss: 0.5178 - accuracy: 0.8131"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "131/250 [==============>...............] - ETA: 0s - loss: 0.5202 - accuracy: 0.8132"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "149/250 [================>.............] - ETA: 0s - loss: 0.5202 - accuracy: 0.8135"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "168/250 [===================>..........] - ETA: 0s - loss: 0.5172 - accuracy: 0.8131"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "187/250 [=====================>........] - ETA: 0s - loss: 0.5192 - accuracy: 0.8127"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "207/250 [=======================>......] - ETA: 0s - loss: 0.5249 - accuracy: 0.8111"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "225/250 [==========================>...] - ETA: 0s - loss: 0.5234 - accuracy: 0.8118"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "245/250 [============================>.] - ETA: 0s - loss: 0.5186 - accuracy: 0.8153"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "250/250 [==============================] - 1s 3ms/step - loss: 0.5179 - accuracy: 0.8163\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/250 [..............................] - ETA: 15s - loss: 0.6443 - accuracy: 0.7500"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 25/250 [==>...........................] - ETA: 0s - loss: 0.5288 - accuracy: 0.8075 "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 49/250 [====>.........................] - ETA: 0s - loss: 0.5457 - accuracy: 0.7966"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 74/250 [=======>......................] - ETA: 0s - loss: 0.5607 - accuracy: 0.7914"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 98/250 [==========>...................] - ETA: 0s - loss: 0.5436 - accuracy: 0.8004"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "121/250 [=============>................] - ETA: 0s - loss: 0.5389 - accuracy: 0.8011"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "143/250 [================>.............] - ETA: 0s - loss: 0.5359 - accuracy: 0.8022"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "166/250 [==================>...........] - ETA: 0s - loss: 0.5356 - accuracy: 0.8027"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "188/250 [=====================>........] - ETA: 0s - loss: 0.5306 - accuracy: 0.8044"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "212/250 [========================>.....] - ETA: 0s - loss: 0.5330 - accuracy: 0.8047"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "236/250 [===========================>..] - ETA: 0s - loss: 0.5321 - accuracy: 0.8056"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "250/250 [==============================] - 1s 2ms/step - loss: 0.5309 - accuracy: 0.8070\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Binary model accuracy: 81.63%\n",
      "Int model accuracy: 80.70%\n"
     ]
    }
   ],
   "source": [
    "binary_loss, binary_accuracy = binary_model.evaluate(binary_test_ds)\n",
    "int_loss, int_accuracy = int_model.evaluate(int_test_ds)\n",
    "\n",
    "print(\"Binary model accuracy: {:2.2%}\".format(binary_accuracy))\n",
    "print(\"Int model accuracy: {:2.2%}\".format(int_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F9dhj8Hey9DS"
   },
   "source": [
    "注：此示例数据集代表了一个相当简单的分类问题。更复杂的数据集和问题会在预处理策略和模型架构上带来微妙但显著的差异。务必尝试不同的超参数和周期来比较各种方法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "h9GaXTsIgP-3"
   },
   "source": [
    "### 导出模型\n",
    "\n",
    "在上面的代码中，您在向模型馈送文本之前对数据集应用了 `tf.keras.layers.TextVectorization`。如果您想让模型能够处理原始字符串（例如，为了简化部署），您可以在模型中包含 `TextVectorization` 层。\n",
    "\n",
    "为此，您可以使用刚刚训练的权重创建一个新模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:11.978846Z",
     "iopub.status.busy": "2022-08-31T05:40:11.978534Z",
     "iopub.status.idle": "2022-08-31T05:40:13.371033Z",
     "shell.execute_reply": "2022-08-31T05:40:13.370230Z"
    },
    "id": "_bRe3KX8gRCX"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/250 [..............................] - ETA: 56s - loss: 0.5008 - accuracy: 0.7500"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 12/250 [>.............................] - ETA: 1s - loss: 0.5363 - accuracy: 0.7917 "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 24/250 [=>............................] - ETA: 1s - loss: 0.5258 - accuracy: 0.8125"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 36/250 [===>..........................] - ETA: 0s - loss: 0.5079 - accuracy: 0.8212"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 48/250 [====>.........................] - ETA: 0s - loss: 0.5283 - accuracy: 0.8105"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 60/250 [======>.......................] - ETA: 0s - loss: 0.5286 - accuracy: 0.8104"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 71/250 [=======>......................] - ETA: 0s - loss: 0.5312 - accuracy: 0.8099"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 83/250 [========>.....................] - ETA: 0s - loss: 0.5282 - accuracy: 0.8114"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 94/250 [==========>...................] - ETA: 0s - loss: 0.5209 - accuracy: 0.8142"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "106/250 [===========>..................] - ETA: 0s - loss: 0.5185 - accuracy: 0.8149"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "118/250 [=============>................] - ETA: 0s - loss: 0.5203 - accuracy: 0.8125"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "130/250 [==============>...............] - ETA: 0s - loss: 0.5202 - accuracy: 0.8139"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "141/250 [===============>..............] - ETA: 0s - loss: 0.5193 - accuracy: 0.8138"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "153/250 [=================>............] - ETA: 0s - loss: 0.5202 - accuracy: 0.8131"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "165/250 [==================>...........] - ETA: 0s - loss: 0.5210 - accuracy: 0.8117"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "177/250 [====================>.........] - ETA: 0s - loss: 0.5168 - accuracy: 0.8141"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "189/250 [=====================>........] - ETA: 0s - loss: 0.5157 - accuracy: 0.8145"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "201/250 [=======================>......] - ETA: 0s - loss: 0.5181 - accuracy: 0.8141"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "213/250 [========================>.....] - ETA: 0s - loss: 0.5203 - accuracy: 0.8132"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "224/250 [=========================>....] - ETA: 0s - loss: 0.5210 - accuracy: 0.8129"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "236/250 [===========================>..] - ETA: 0s - loss: 0.5199 - accuracy: 0.8137"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "248/250 [============================>.] - ETA: 0s - loss: 0.5178 - accuracy: 0.8159"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "250/250 [==============================] - 1s 4ms/step - loss: 0.5179 - accuracy: 0.8163\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 81.63%\n"
     ]
    }
   ],
   "source": [
    "export_model = tf.keras.Sequential(\n",
    "    [binary_vectorize_layer, binary_model,\n",
    "     layers.Activation('sigmoid')])\n",
    "\n",
    "export_model.compile(\n",
    "    loss=losses.SparseCategoricalCrossentropy(from_logits=False),\n",
    "    optimizer='adam',\n",
    "    metrics=['accuracy'])\n",
    "\n",
    "# Test it with `raw_test_ds`, which yields raw strings\n",
    "loss, accuracy = export_model.evaluate(raw_test_ds)\n",
    "print(\"Accuracy: {:2.2%}\".format(binary_accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m2eqTVBP4DUN"
   },
   "source": [
    "现在，您的模型可以将原始字符串作为输入，并使用 `Model.predict` 预测每个标签的得分。定义一个函数来查找得分最高的标签："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:13.374780Z",
     "iopub.status.busy": "2022-08-31T05:40:13.374517Z",
     "iopub.status.idle": "2022-08-31T05:40:13.378503Z",
     "shell.execute_reply": "2022-08-31T05:40:13.377781Z"
    },
    "id": "GU53uRXz45iO"
   },
   "outputs": [],
   "source": [
    "def get_string_labels(predicted_scores_batch):\n",
    "  predicted_int_labels = tf.math.argmax(predicted_scores_batch, axis=1)\n",
    "  predicted_labels = tf.gather(raw_train_ds.class_names, predicted_int_labels)\n",
    "  return predicted_labels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yqnWc7Nn5eou"
   },
   "source": [
    "### 在新数据上运行推断"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:13.382262Z",
     "iopub.status.busy": "2022-08-31T05:40:13.381744Z",
     "iopub.status.idle": "2022-08-31T05:40:13.543376Z",
     "shell.execute_reply": "2022-08-31T05:40:13.542628Z"
    },
    "id": "BOR2MupW1_zS"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Question:  how do I extract keys from a dict into a list?\n",
      "Predicted label:  b'python'\n",
      "Question:  debug public static void main(string[] args) {...}\n",
      "Predicted label:  b'java'\n"
     ]
    }
   ],
   "source": [
    "inputs = [\n",
    "    \"how do I extract keys from a dict into a list?\",  # 'python'\n",
    "    \"debug public static void main(string[] args) {...}\",  # 'java'\n",
    "]\n",
    "predicted_scores = export_model.predict(inputs)\n",
    "predicted_labels = get_string_labels(predicted_scores)\n",
    "for input, label in zip(inputs, predicted_labels):\n",
    "  print(\"Question: \", input)\n",
    "  print(\"Predicted label: \", label.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0QDVfii_4slI"
   },
   "source": [
    "将文本预处理逻辑包含在模型中后，您可以导出用于生产的模型，从而简化部署并降低[训练/测试偏差](https://developers.google.com/machine-learning/guides/rules-of-ml#training-serving_skew)的可能性。\n",
    "\n",
    "在选择应用 `tf.keras.layers.TextVectorization` 层的位置时，需要注意性能差异。在模型之外使用它可以让您在 GPU 上训练时进行异步 CPU 处理和数据缓冲。因此，如果您在 GPU 上训练模型，您应该在开发模型时使用此选项以获得最佳性能，然后在准备好部署时进行切换，在模型中包含 `TextVectorization` 层。\n",
    "\n",
    "请参阅[保存和加载模型](../keras/save_and_load.ipynb)教程，详细了解如何保存模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "p4cvuFzavTRy"
   },
   "source": [
    "## 例 2：预测《伊利亚特》翻译的作者\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fOlJ22508RIe"
   },
   "source": [
    "下面提供了一个使用 `tf.data.TextLineDataset` 从文本文件中加载样本，以及使用 [TensorFlow Text](https://tensorflow.google.cn/text) 预处理数据的示例。您将使用同一作品（荷马的《伊利亚特》）的三种不同英语翻译，训练一个模型来识别给定单行文本的译者。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-pCgKbOSk7kU"
   },
   "source": [
    "### 下载并探索数据集\n",
    "\n",
    "三个译本的文本来自：\n",
    "\n",
    "- [William Cowper](https://en.wikipedia.org/wiki/William_Cowper)：[文本](https://storage.googleapis.com/download.tensorflow.org/data/illiad/cowper.txt)\n",
    "- [Edward, Earl of Derby](https://en.wikipedia.org/wiki/Edward_Smith-Stanley,_14th_Earl_of_Derby)：[文本](https://storage.googleapis.com/download.tensorflow.org/data/illiad/derby.txt)\n",
    "- [Samuel Butler](https://en.wikipedia.org/wiki/Samuel_Butler_%28novelist%29)：[文本](https://storage.googleapis.com/download.tensorflow.org/data/illiad/butler.txt)\n",
    "\n",
    "本教程中使用的文本文件经历了一些典型的预处理任务，例如移除文档页眉和页脚、行号和章节标题。\n",
    "\n",
    "将这些稍微改动过的文件下载到本地："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:13.548162Z",
     "iopub.status.busy": "2022-08-31T05:40:13.547538Z",
     "iopub.status.idle": "2022-08-31T05:40:13.687700Z",
     "shell.execute_reply": "2022-08-31T05:40:13.686895Z"
    },
    "id": "4YlKQthEYlFw"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/illiad/cowper.txt\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      " 16384/815980 [..............................] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "819200/815980 [==============================] - 0s 0us/step\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "827392/815980 [==============================] - 0s 0us/step\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/illiad/derby.txt\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      " 16384/809730 [..............................] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "811008/809730 [==============================] - 0s 0us/step\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "819200/809730 [==============================] - 0s 0us/step\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/illiad/butler.txt\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      " 16384/807992 [..............................] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "811008/807992 [==============================] - 0s 0us/step\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "819200/807992 [==============================] - 0s 0us/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[PosixPath('/home/kbuilder/.keras/datasets/kandinsky5.jpg'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/spa-eng.zip'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/fashion-mnist'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/194px-New_East_River_Bridge_from_Brooklyn_det.4a09796u.jpg'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/facades'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/butler.txt'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/flower_photos'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/train.csv'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/facades.tar.gz'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/derby.txt'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/spa-eng'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/320px-Felis_catus-cat_on_snow.jpg'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/HIGGS.csv.gz'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/flower_photos.tar.gz'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/mnist.npz'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/YellowLabradorLooking_new.jpg'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/shakespeare.txt'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/cifar-10-batches-py.tar.gz'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/cowper.txt'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/iris_test.csv'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/cifar-10-batches-py'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/cats_and_dogs_filtered'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/eval.csv'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/cats_and_dogs.zip'),\n",
       " PosixPath('/home/kbuilder/.keras/datasets/iris_training.csv')]"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "DIRECTORY_URL = 'https://storage.googleapis.com/download.tensorflow.org/data/illiad/'\n",
    "FILE_NAMES = ['cowper.txt', 'derby.txt', 'butler.txt']\n",
    "\n",
    "for name in FILE_NAMES:\n",
    "  text_dir = utils.get_file(name, origin=DIRECTORY_URL + name)\n",
    "\n",
    "parent_dir = pathlib.Path(text_dir).parent\n",
    "list(parent_dir.iterdir())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M8PHK5J_cXE5"
   },
   "source": [
    "### 加载数据集\n",
    "\n",
    "以前，使用 `tf.keras.utils.text_dataset_from_directory` 时，文件的所有内容都会被视为单个样本。在这里，您将使用 `tf.data.TextLineDataset`，它旨在从文本文件创建 `tf.data.Dataset`，其中每个样本都是原始文件中的一行文本。`TextLineDataset` 对于主要基于行的文本数据（例如，诗歌或错误日志）非常有用。\n",
    "\n",
    "遍历这些文件，将每个文件加载到自己的数据集中。每个样本都需要单独加标签，因此请使用 `Dataset.map` 为每个样本应用标签添加器功能。这将遍历数据集中的每个样本，同时返回 (`example, label`) 对。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:13.691569Z",
     "iopub.status.busy": "2022-08-31T05:40:13.691320Z",
     "iopub.status.idle": "2022-08-31T05:40:13.695126Z",
     "shell.execute_reply": "2022-08-31T05:40:13.694257Z"
    },
    "id": "YIIWIdPXgk7I"
   },
   "outputs": [],
   "source": [
    "def labeler(example, index):\n",
    "  return example, tf.cast(index, tf.int64)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:13.698542Z",
     "iopub.status.busy": "2022-08-31T05:40:13.698035Z",
     "iopub.status.idle": "2022-08-31T05:40:13.758815Z",
     "shell.execute_reply": "2022-08-31T05:40:13.757970Z"
    },
    "id": "8Ajx7AmZnEg3"
   },
   "outputs": [],
   "source": [
    "labeled_data_sets = []\n",
    "\n",
    "for i, file_name in enumerate(FILE_NAMES):\n",
    "  lines_dataset = tf.data.TextLineDataset(str(parent_dir/file_name))\n",
    "  labeled_dataset = lines_dataset.map(lambda ex: labeler(ex, i))\n",
    "  labeled_data_sets.append(labeled_dataset)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wPOsVK1e9NGM"
   },
   "source": [
    "接下来，您将使用 `Dataset.concatenate` 将这些带标签的数据集组合到一个数据集中，并使用 `Dataset.shuffle` 打乱其顺序：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:13.762571Z",
     "iopub.status.busy": "2022-08-31T05:40:13.762349Z",
     "iopub.status.idle": "2022-08-31T05:40:13.765561Z",
     "shell.execute_reply": "2022-08-31T05:40:13.764879Z"
    },
    "id": "6jAeYkTIi9-2"
   },
   "outputs": [],
   "source": [
    "BUFFER_SIZE = 50000\n",
    "BATCH_SIZE = 64\n",
    "VALIDATION_SIZE = 5000"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:13.768610Z",
     "iopub.status.busy": "2022-08-31T05:40:13.768368Z",
     "iopub.status.idle": "2022-08-31T05:40:13.773847Z",
     "shell.execute_reply": "2022-08-31T05:40:13.773171Z"
    },
    "id": "Qd544E-Sh63L"
   },
   "outputs": [],
   "source": [
    "all_labeled_data = labeled_data_sets[0]\n",
    "for labeled_dataset in labeled_data_sets[1:]:\n",
    "  all_labeled_data = all_labeled_data.concatenate(labeled_dataset)\n",
    "\n",
    "all_labeled_data = all_labeled_data.shuffle(\n",
    "    BUFFER_SIZE, reshuffle_each_iteration=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r4JEHrJXeG5k"
   },
   "source": [
    "像以前一样打印出几个样本。数据集尚未经过批处理，因此 `all_labeled_data` 中的每个条目都对应一个数据点："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:13.777420Z",
     "iopub.status.busy": "2022-08-31T05:40:13.776832Z",
     "iopub.status.idle": "2022-08-31T05:40:14.342758Z",
     "shell.execute_reply": "2022-08-31T05:40:14.341991Z"
    },
    "id": "gywKlN0xh6u5"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentence:  b'And like two lofty pines in death they lay.'\n",
      "Label: 1\n",
      "Sentence:  b'Men and robed matrons, who shall seek the Gods'\n",
      "Label: 0\n",
      "Sentence:  b'So vaunted he, but Juno with disdain'\n",
      "Label: 0\n",
      "Sentence:  b'First turning, slew the mighty Bathycles,'\n",
      "Label: 1\n",
      "Sentence:  b'flames, while the women and children are carried into captivity; when'\n",
      "Label: 2\n",
      "Sentence:  b'finished. When they had heaped up the barrow they went back again into'\n",
      "Label: 2\n",
      "Sentence:  b\"He said; and from th' applauding ranks of Greece\"\n",
      "Label: 1\n",
      "Sentence:  b'the fate of Hector; at length Minerva descends to the aid of Achilles.'\n",
      "Label: 1\n",
      "Sentence:  b'sit down and keep your eyes on the horses; they are speeding towards'\n",
      "Label: 2\n",
      "Sentence:  b'dearest of his friends has fallen. But I can see not a man among the'\n",
      "Label: 2\n"
     ]
    }
   ],
   "source": [
    "for text, label in all_labeled_data.take(10):\n",
    "  print(\"Sentence: \", text.numpy())\n",
    "  print(\"Label:\", label.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5rrpU2_sfDh0"
   },
   "source": [
    "### 准备用于训练的数据集\n",
    "\n",
    "现在，将不再使用 `tf.keras.layers.TextVectorization` 来预处理文本数据集，而是使用 TensorFlow Text API 对数据进行标准化和词例化、构建词汇表并使用 `tf.lookup.StaticVocabularyTable` 将词例映射到整数以馈送给模型。（详细了解 [TensorFlow Text](https://tensorflow.google.cn/text)）。\n",
    "\n",
    "定义一个将文本转换为小写并对其进行词例化的函数：\n",
    "\n",
    "- TensorFlow Text 提供各种分词器。在此示例中，您将使用 `text.UnicodeScriptTokenizer` 对数据集进行词例化。\n",
    "- 您将使用 `Dataset.map` 将词例化应用于数据集。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:14.347196Z",
     "iopub.status.busy": "2022-08-31T05:40:14.346601Z",
     "iopub.status.idle": "2022-08-31T05:40:14.350522Z",
     "shell.execute_reply": "2022-08-31T05:40:14.349816Z"
    },
    "id": "v4DpQW-Y12rm"
   },
   "outputs": [],
   "source": [
    "tokenizer = tf_text.UnicodeScriptTokenizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:14.353989Z",
     "iopub.status.busy": "2022-08-31T05:40:14.353457Z",
     "iopub.status.idle": "2022-08-31T05:40:14.357053Z",
     "shell.execute_reply": "2022-08-31T05:40:14.356398Z"
    },
    "id": "pz8xEj0ugu51"
   },
   "outputs": [],
   "source": [
    "def tokenize(text, unused_label):\n",
    "  lower_case = tf_text.case_fold_utf8(text)\n",
    "  return tokenizer.tokenize(lower_case)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:14.360470Z",
     "iopub.status.busy": "2022-08-31T05:40:14.360225Z",
     "iopub.status.idle": "2022-08-31T05:40:16.192868Z",
     "shell.execute_reply": "2022-08-31T05:40:16.192053Z"
    },
    "id": "vzUrAzOq31QL"
   },
   "outputs": [],
   "source": [
    "tokenized_ds = all_labeled_data.map(tokenize)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jx4Q2i8XLV7o"
   },
   "source": [
    "您可以遍历数据集并打印出一些词例化的样本：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:16.197468Z",
     "iopub.status.busy": "2022-08-31T05:40:16.197178Z",
     "iopub.status.idle": "2022-08-31T05:40:17.360465Z",
     "shell.execute_reply": "2022-08-31T05:40:17.359556Z"
    },
    "id": "g2mkWri7LiGq"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tokens:  [b'and' b'like' b'two' b'lofty' b'pines' b'in' b'death' b'they' b'lay'\n",
      " b'.']\n",
      "Tokens:  [b'men' b'and' b'robed' b'matrons' b',' b'who' b'shall' b'seek' b'the'\n",
      " b'gods']\n",
      "Tokens:  [b'so' b'vaunted' b'he' b',' b'but' b'juno' b'with' b'disdain']\n",
      "Tokens:  [b'first' b'turning' b',' b'slew' b'the' b'mighty' b'bathycles' b',']\n",
      "Tokens:  [b'flames' b',' b'while' b'the' b'women' b'and' b'children' b'are'\n",
      " b'carried' b'into' b'captivity' b';' b'when']\n"
     ]
    }
   ],
   "source": [
    "for text_batch in tokenized_ds.take(5):\n",
    "  print(\"Tokens: \", text_batch.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JPd4PsskJ_Xt"
   },
   "source": [
    "接下来，您将通过按频率对词例进行排序并保留顶部 `VOCAB_SIZE` 词例来构建词汇表："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:17.364684Z",
     "iopub.status.busy": "2022-08-31T05:40:17.364084Z",
     "iopub.status.idle": "2022-08-31T05:40:24.687562Z",
     "shell.execute_reply": "2022-08-31T05:40:24.686686Z"
    },
    "id": "YkHtbGnDh6mg"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vocab size:  10000\n",
      "First five vocab entries: [b',', b'the', b'and', b\"'\", b'of']\n"
     ]
    }
   ],
   "source": [
    "tokenized_ds = configure_dataset(tokenized_ds)\n",
    "\n",
    "vocab_dict = collections.defaultdict(lambda: 0)\n",
    "for toks in tokenized_ds.as_numpy_iterator():\n",
    "  for tok in toks:\n",
    "    vocab_dict[tok] += 1\n",
    "\n",
    "vocab = sorted(vocab_dict.items(), key=lambda x: x[1], reverse=True)\n",
    "vocab = [token for token, count in vocab]\n",
    "vocab = vocab[:VOCAB_SIZE]\n",
    "vocab_size = len(vocab)\n",
    "print(\"Vocab size: \", vocab_size)\n",
    "print(\"First five vocab entries:\", vocab[:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PyKSsaNAKi17"
   },
   "source": [
    "要将词例转换为整数，请使用 `vocab` 集创建 `tf.lookup.StaticVocabularyTable`。您将词例映射到 [`2`, `vocab_size + 2`] 范围内的整数。与 `TextVectorization` 层一样，保留 `0` 表示填充，保留 `1` 表示词汇表外 (OOV) 词例。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:24.692361Z",
     "iopub.status.busy": "2022-08-31T05:40:24.691681Z",
     "iopub.status.idle": "2022-08-31T05:40:24.701111Z",
     "shell.execute_reply": "2022-08-31T05:40:24.700451Z"
    },
    "id": "kCBo2yFHD7y6"
   },
   "outputs": [],
   "source": [
    "keys = vocab\n",
    "values = range(2, len(vocab) + 2)  # Reserve `0` for padding, `1` for OOV tokens.\n",
    "\n",
    "init = tf.lookup.KeyValueTensorInitializer(\n",
    "    keys, values, key_dtype=tf.string, value_dtype=tf.int64)\n",
    "\n",
    "num_oov_buckets = 1\n",
    "vocab_table = tf.lookup.StaticVocabularyTable(init, num_oov_buckets)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Z5F-EiBpOADE"
   },
   "source": [
    "最后，定义一个函数来使用分词器和查找表对数据集进行标准化、词例化和向量化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:24.705174Z",
     "iopub.status.busy": "2022-08-31T05:40:24.704520Z",
     "iopub.status.idle": "2022-08-31T05:40:24.708199Z",
     "shell.execute_reply": "2022-08-31T05:40:24.707552Z"
    },
    "id": "HcIQ7LOTh6eT"
   },
   "outputs": [],
   "source": [
    "def preprocess_text(text, label):\n",
    "  standardized = tf_text.case_fold_utf8(text)\n",
    "  tokenized = tokenizer.tokenize(standardized)\n",
    "  vectorized = vocab_table.lookup(tokenized)\n",
    "  return vectorized, label"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "v6S5Qyabi-vo"
   },
   "source": [
    "您可以在单个样本上尝试此操作并打印输出："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:24.712002Z",
     "iopub.status.busy": "2022-08-31T05:40:24.711389Z",
     "iopub.status.idle": "2022-08-31T05:40:25.410681Z",
     "shell.execute_reply": "2022-08-31T05:40:25.409901Z"
    },
    "id": "jgxPZaxUuTbk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sentence:  b'And like two lofty pines in death they lay.'\n",
      "Vectorized sentence:  [   4  158  104  409 7806   13  134   27  249    7]\n"
     ]
    }
   ],
   "source": [
    "example_text, example_label = next(iter(all_labeled_data))\n",
    "print(\"Sentence: \", example_text.numpy())\n",
    "vectorized_text, example_label = preprocess_text(example_text, example_label)\n",
    "print(\"Vectorized sentence: \", vectorized_text.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "p9qHM0v8k_Mg"
   },
   "source": [
    "现在，使用 `Dataset.map` 在数据集上运行预处理函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:25.414674Z",
     "iopub.status.busy": "2022-08-31T05:40:25.414398Z",
     "iopub.status.idle": "2022-08-31T05:40:26.832153Z",
     "shell.execute_reply": "2022-08-31T05:40:26.831409Z"
    },
    "id": "KmQVsAgJ-RM0"
   },
   "outputs": [],
   "source": [
    "all_encoded_data = all_labeled_data.map(preprocess_text)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_YZToSXSm0qr"
   },
   "source": [
    "### 将数据集拆分为训练集和测试集\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "itxIJwkrUXgv"
   },
   "source": [
    "Keras `TextVectorization` 层还会对向量化数据进行批处理和填充。填充是必需的，因为批次内的样本需要具有相同的大小和形状，但这些数据集中的样本并非全部相同 – 每行文本具有不同数量的单词。\n",
    "\n",
    "`tf.data.Dataset` 支持拆分和填充批次数据集："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:26.837314Z",
     "iopub.status.busy": "2022-08-31T05:40:26.836680Z",
     "iopub.status.idle": "2022-08-31T05:40:26.841986Z",
     "shell.execute_reply": "2022-08-31T05:40:26.841425Z"
    },
    "id": "r-rmbijQh6bf"
   },
   "outputs": [],
   "source": [
    "train_data = all_encoded_data.skip(VALIDATION_SIZE).shuffle(BUFFER_SIZE)\n",
    "validation_data = all_encoded_data.take(VALIDATION_SIZE)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:26.845671Z",
     "iopub.status.busy": "2022-08-31T05:40:26.845189Z",
     "iopub.status.idle": "2022-08-31T05:40:26.852634Z",
     "shell.execute_reply": "2022-08-31T05:40:26.852088Z"
    },
    "id": "qTP0IwHBCn0Q"
   },
   "outputs": [],
   "source": [
    "train_data = train_data.padded_batch(BATCH_SIZE)\n",
    "validation_data = validation_data.padded_batch(BATCH_SIZE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m-wmFq8uW1zS"
   },
   "source": [
    "现在，`validation_data` 和 `train_data` 不是 (`example, label`) 对的集合，而是批次的集合。每个批次都是一对表示为数组的（*许多样本*、*许多标签*）。\n",
    "\n",
    "为了说明这一点："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:26.856109Z",
     "iopub.status.busy": "2022-08-31T05:40:26.855683Z",
     "iopub.status.idle": "2022-08-31T05:40:28.078522Z",
     "shell.execute_reply": "2022-08-31T05:40:28.077788Z"
    },
    "id": "kMslWfuwoqpB"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text batch shape:  (64, 17)\n",
      "Label batch shape:  (64,)\n",
      "First text example:  tf.Tensor(\n",
      "[   4  158  104  409 7806   13  134   27  249    7    0    0    0    0\n",
      "    0    0    0], shape=(17,), dtype=int64)\n",
      "First label example:  tf.Tensor(1, shape=(), dtype=int64)\n"
     ]
    }
   ],
   "source": [
    "sample_text, sample_labels = next(iter(validation_data))\n",
    "print(\"Text batch shape: \", sample_text.shape)\n",
    "print(\"Label batch shape: \", sample_labels.shape)\n",
    "print(\"First text example: \", sample_text[0])\n",
    "print(\"First label example: \", sample_labels[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UI4I6_Sa0vWu"
   },
   "source": [
    "由于您将 `0` 用于填充，将 `1` 用于词汇外 (OOV) 词例，词汇量增加了两倍："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:28.082343Z",
     "iopub.status.busy": "2022-08-31T05:40:28.081981Z",
     "iopub.status.idle": "2022-08-31T05:40:28.085313Z",
     "shell.execute_reply": "2022-08-31T05:40:28.084679Z"
    },
    "id": "u21LlkO8QGRX"
   },
   "outputs": [],
   "source": [
    "vocab_size += 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "h44Ox11OYLP-"
   },
   "source": [
    "像以前一样配置数据集以提高性能："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:28.088771Z",
     "iopub.status.busy": "2022-08-31T05:40:28.088313Z",
     "iopub.status.idle": "2022-08-31T05:40:28.093154Z",
     "shell.execute_reply": "2022-08-31T05:40:28.092496Z"
    },
    "id": "BpT0b_7mYRXV"
   },
   "outputs": [],
   "source": [
    "train_data = configure_dataset(train_data)\n",
    "validation_data = configure_dataset(validation_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "K8SUhGFNsmRi"
   },
   "source": [
    "### 训练模型\n",
    "\n",
    "您可以像以前一样在此数据集上训练模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:40:28.096532Z",
     "iopub.status.busy": "2022-08-31T05:40:28.096027Z",
     "iopub.status.idle": "2022-08-31T05:41:03.826959Z",
     "shell.execute_reply": "2022-08-31T05:41:03.826198Z"
    },
    "id": "QJgI1pow2YR9"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "      1/Unknown - 23s 23s/step - loss: 1.1031 - accuracy: 0.2500"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     11/Unknown - 23s 5ms/step - loss: 1.0758 - accuracy: 0.4148"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     26/Unknown - 23s 4ms/step - loss: 1.0418 - accuracy: 0.4375"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     42/Unknown - 23s 4ms/step - loss: 1.0142 - accuracy: 0.4539"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     59/Unknown - 23s 4ms/step - loss: 0.9823 - accuracy: 0.5032"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     76/Unknown - 23s 3ms/step - loss: 0.9414 - accuracy: 0.5411"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     94/Unknown - 23s 3ms/step - loss: 0.9065 - accuracy: 0.5710"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    112/Unknown - 23s 3ms/step - loss: 0.8707 - accuracy: 0.5882"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    130/Unknown - 23s 3ms/step - loss: 0.8422 - accuracy: 0.6058"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    147/Unknown - 23s 3ms/step - loss: 0.8181 - accuracy: 0.6184"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    164/Unknown - 23s 3ms/step - loss: 0.7926 - accuracy: 0.6332"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    181/Unknown - 23s 3ms/step - loss: 0.7708 - accuracy: 0.6461"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    199/Unknown - 23s 3ms/step - loss: 0.7464 - accuracy: 0.6588"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    217/Unknown - 23s 3ms/step - loss: 0.7262 - accuracy: 0.6699"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    232/Unknown - 23s 3ms/step - loss: 0.7135 - accuracy: 0.6766"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    249/Unknown - 23s 3ms/step - loss: 0.6999 - accuracy: 0.6834"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    267/Unknown - 23s 3ms/step - loss: 0.6853 - accuracy: 0.6914"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    285/Unknown - 23s 3ms/step - loss: 0.6728 - accuracy: 0.6976"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    302/Unknown - 23s 3ms/step - loss: 0.6612 - accuracy: 0.7038"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    319/Unknown - 24s 3ms/step - loss: 0.6502 - accuracy: 0.7094"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    336/Unknown - 24s 3ms/step - loss: 0.6410 - accuracy: 0.7144"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    354/Unknown - 24s 3ms/step - loss: 0.6322 - accuracy: 0.7191"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    372/Unknown - 24s 3ms/step - loss: 0.6223 - accuracy: 0.7237"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    390/Unknown - 24s 3ms/step - loss: 0.6145 - accuracy: 0.7270"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    408/Unknown - 24s 3ms/step - loss: 0.6058 - accuracy: 0.7308"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    426/Unknown - 24s 3ms/step - loss: 0.5980 - accuracy: 0.7345"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    444/Unknown - 24s 3ms/step - loss: 0.5916 - accuracy: 0.7374"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    462/Unknown - 24s 3ms/step - loss: 0.5845 - accuracy: 0.7405"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    480/Unknown - 24s 3ms/step - loss: 0.5775 - accuracy: 0.7441"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    498/Unknown - 24s 3ms/step - loss: 0.5713 - accuracy: 0.7468"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    515/Unknown - 24s 3ms/step - loss: 0.5664 - accuracy: 0.7497"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    533/Unknown - 24s 3ms/step - loss: 0.5608 - accuracy: 0.7520"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    551/Unknown - 24s 3ms/step - loss: 0.5561 - accuracy: 0.7543"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    569/Unknown - 24s 3ms/step - loss: 0.5513 - accuracy: 0.7563"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    586/Unknown - 24s 3ms/step - loss: 0.5458 - accuracy: 0.7590"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    603/Unknown - 24s 3ms/step - loss: 0.5409 - accuracy: 0.7610"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    621/Unknown - 24s 3ms/step - loss: 0.5364 - accuracy: 0.7632"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    639/Unknown - 24s 3ms/step - loss: 0.5326 - accuracy: 0.7652"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    656/Unknown - 25s 3ms/step - loss: 0.5293 - accuracy: 0.7666"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    673/Unknown - 25s 3ms/step - loss: 0.5261 - accuracy: 0.7682"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    691/Unknown - 25s 3ms/step - loss: 0.5221 - accuracy: 0.7701"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "697/697 [==============================] - 29s 9ms/step - loss: 0.5210 - accuracy: 0.7706 - val_loss: 0.3762 - val_accuracy: 0.8382\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 2/3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/697 [..............................] - ETA: 4s - loss: 0.3529 - accuracy: 0.8438"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 17/697 [..............................] - ETA: 2s - loss: 0.3577 - accuracy: 0.8493"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 35/697 [>.............................] - ETA: 2s - loss: 0.3425 - accuracy: 0.8580"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 53/697 [=>............................] - ETA: 1s - loss: 0.3435 - accuracy: 0.8588"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 71/697 [==>...........................] - ETA: 1s - loss: 0.3414 - accuracy: 0.8614"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 89/697 [==>...........................] - ETA: 1s - loss: 0.3383 - accuracy: 0.8618"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "107/697 [===>..........................] - ETA: 1s - loss: 0.3381 - accuracy: 0.8607"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "125/697 [====>.........................] - ETA: 1s - loss: 0.3368 - accuracy: 0.8615"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "143/697 [=====>........................] - ETA: 1s - loss: 0.3348 - accuracy: 0.8629"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "161/697 [=====>........................] - ETA: 1s - loss: 0.3308 - accuracy: 0.8663"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "179/697 [======>.......................] - ETA: 1s - loss: 0.3287 - accuracy: 0.8674"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "196/697 [=======>......................] - ETA: 1s - loss: 0.3226 - accuracy: 0.8702"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "214/697 [========>.....................] - ETA: 1s - loss: 0.3195 - accuracy: 0.8722"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "232/697 [========>.....................] - ETA: 1s - loss: 0.3180 - accuracy: 0.8724"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "250/697 [=========>....................] - ETA: 1s - loss: 0.3168 - accuracy: 0.8726"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "268/697 [==========>...................] - ETA: 1s - loss: 0.3143 - accuracy: 0.8738"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "286/697 [===========>..................] - ETA: 1s - loss: 0.3134 - accuracy: 0.8748"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "304/697 [============>.................] - ETA: 1s - loss: 0.3113 - accuracy: 0.8759"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "322/697 [============>.................] - ETA: 1s - loss: 0.3100 - accuracy: 0.8763"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "339/697 [=============>................] - ETA: 1s - loss: 0.3101 - accuracy: 0.8760"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "357/697 [==============>...............] - ETA: 0s - loss: 0.3097 - accuracy: 0.8760"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "374/697 [===============>..............] - ETA: 0s - loss: 0.3072 - accuracy: 0.8768"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "392/697 [===============>..............] - ETA: 0s - loss: 0.3064 - accuracy: 0.8767"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "410/697 [================>.............] - ETA: 0s - loss: 0.3044 - accuracy: 0.8776"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "427/697 [=================>............] - ETA: 0s - loss: 0.3031 - accuracy: 0.8782"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "444/697 [==================>...........] - ETA: 0s - loss: 0.3026 - accuracy: 0.8784"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "461/697 [==================>...........] - ETA: 0s - loss: 0.3010 - accuracy: 0.8785"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "478/697 [===================>..........] - ETA: 0s - loss: 0.2997 - accuracy: 0.8790"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "495/697 [====================>.........] - ETA: 0s - loss: 0.2985 - accuracy: 0.8794"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "512/697 [=====================>........] - ETA: 0s - loss: 0.2975 - accuracy: 0.8798"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "529/697 [=====================>........] - ETA: 0s - loss: 0.2962 - accuracy: 0.8803"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "546/697 [======================>.......] - ETA: 0s - loss: 0.2953 - accuracy: 0.8808"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "563/697 [=======================>......] - ETA: 0s - loss: 0.2946 - accuracy: 0.8813"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "581/697 [========================>.....] - ETA: 0s - loss: 0.2935 - accuracy: 0.8814"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "599/697 [========================>.....] - ETA: 0s - loss: 0.2921 - accuracy: 0.8819"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "616/697 [=========================>....] - ETA: 0s - loss: 0.2906 - accuracy: 0.8823"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "632/697 [==========================>...] - ETA: 0s - loss: 0.2900 - accuracy: 0.8827"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "650/697 [==========================>...] - ETA: 0s - loss: 0.2897 - accuracy: 0.8828"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "668/697 [===========================>..] - ETA: 0s - loss: 0.2891 - accuracy: 0.8829"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "686/697 [============================>.] - ETA: 0s - loss: 0.2889 - accuracy: 0.8829"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "697/697 [==============================] - 3s 4ms/step - loss: 0.2883 - accuracy: 0.8832 - val_loss: 0.3643 - val_accuracy: 0.8482\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 3/3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/697 [..............................] - ETA: 4s - loss: 0.2720 - accuracy: 0.8750"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 18/697 [..............................] - ETA: 2s - loss: 0.2456 - accuracy: 0.9019"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 36/697 [>.............................] - ETA: 1s - loss: 0.2363 - accuracy: 0.9067"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 54/697 [=>............................] - ETA: 1s - loss: 0.2388 - accuracy: 0.9054"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 72/697 [==>...........................] - ETA: 1s - loss: 0.2338 - accuracy: 0.9082"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 90/697 [==>...........................] - ETA: 1s - loss: 0.2301 - accuracy: 0.9097"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "108/697 [===>..........................] - ETA: 1s - loss: 0.2281 - accuracy: 0.9103"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "126/697 [====>.........................] - ETA: 1s - loss: 0.2288 - accuracy: 0.9110"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "144/697 [=====>........................] - ETA: 1s - loss: 0.2278 - accuracy: 0.9112"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "162/697 [=====>........................] - ETA: 1s - loss: 0.2249 - accuracy: 0.9126"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "180/697 [======>.......................] - ETA: 1s - loss: 0.2239 - accuracy: 0.9135"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "198/697 [=======>......................] - ETA: 1s - loss: 0.2189 - accuracy: 0.9151"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "216/697 [========>.....................] - ETA: 1s - loss: 0.2175 - accuracy: 0.9165"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "234/697 [=========>....................] - ETA: 1s - loss: 0.2176 - accuracy: 0.9161"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "251/697 [=========>....................] - ETA: 1s - loss: 0.2167 - accuracy: 0.9165"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "268/697 [==========>...................] - ETA: 1s - loss: 0.2149 - accuracy: 0.9170"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "286/697 [===========>..................] - ETA: 1s - loss: 0.2145 - accuracy: 0.9176"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "304/697 [============>.................] - ETA: 1s - loss: 0.2126 - accuracy: 0.9187"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "322/697 [============>.................] - ETA: 1s - loss: 0.2119 - accuracy: 0.9191"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "339/697 [=============>................] - ETA: 1s - loss: 0.2116 - accuracy: 0.9192"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "357/697 [==============>...............] - ETA: 0s - loss: 0.2119 - accuracy: 0.9186"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "375/697 [===============>..............] - ETA: 0s - loss: 0.2103 - accuracy: 0.9189"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "393/697 [===============>..............] - ETA: 0s - loss: 0.2098 - accuracy: 0.9190"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "411/697 [================>.............] - ETA: 0s - loss: 0.2080 - accuracy: 0.9199"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "429/697 [=================>............] - ETA: 0s - loss: 0.2069 - accuracy: 0.9204"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "447/697 [==================>...........] - ETA: 0s - loss: 0.2062 - accuracy: 0.9208"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "465/697 [===================>..........] - ETA: 0s - loss: 0.2049 - accuracy: 0.9210"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "483/697 [===================>..........] - ETA: 0s - loss: 0.2045 - accuracy: 0.9210"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "501/697 [====================>.........] - ETA: 0s - loss: 0.2035 - accuracy: 0.9217"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "519/697 [=====================>........] - ETA: 0s - loss: 0.2026 - accuracy: 0.9222"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "537/697 [======================>.......] - ETA: 0s - loss: 0.2019 - accuracy: 0.9224"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "555/697 [======================>.......] - ETA: 0s - loss: 0.2010 - accuracy: 0.9230"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "573/697 [=======================>......] - ETA: 0s - loss: 0.2006 - accuracy: 0.9232"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "591/697 [========================>.....] - ETA: 0s - loss: 0.2002 - accuracy: 0.9234"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "609/697 [=========================>....] - ETA: 0s - loss: 0.1983 - accuracy: 0.9241"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "627/697 [=========================>....] - ETA: 0s - loss: 0.1979 - accuracy: 0.9242"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "645/697 [==========================>...] - ETA: 0s - loss: 0.1977 - accuracy: 0.9243"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "663/697 [===========================>..] - ETA: 0s - loss: 0.1971 - accuracy: 0.9245"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "681/697 [============================>.] - ETA: 0s - loss: 0.1967 - accuracy: 0.9247"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "697/697 [==============================] - 3s 4ms/step - loss: 0.1962 - accuracy: 0.9250 - val_loss: 0.3924 - val_accuracy: 0.8470\n"
     ]
    }
   ],
   "source": [
    "model = create_model(vocab_size=vocab_size, num_labels=3)\n",
    "\n",
    "model.compile(\n",
    "    optimizer='adam',\n",
    "    loss=losses.SparseCategoricalCrossentropy(from_logits=True),\n",
    "    metrics=['accuracy'])\n",
    "\n",
    "history = model.fit(train_data, validation_data=validation_data, epochs=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:03.831455Z",
     "iopub.status.busy": "2022-08-31T05:41:03.831151Z",
     "iopub.status.idle": "2022-08-31T05:41:04.650591Z",
     "shell.execute_reply": "2022-08-31T05:41:04.649897Z"
    },
    "id": "KTPCYf_Jh6TH"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "      1/Unknown - 1s 639ms/step - loss: 0.2399 - accuracy: 0.8906"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     28/Unknown - 1s 2ms/step - loss: 0.3763 - accuracy: 0.8477  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     56/Unknown - 1s 2ms/step - loss: 0.3884 - accuracy: 0.8504"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "79/79 [==============================] - 1s 2ms/step - loss: 0.3924 - accuracy: 0.8470\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  0.3924245238304138\n",
      "Accuracy: 84.70%\n"
     ]
    }
   ],
   "source": [
    "loss, accuracy = model.evaluate(validation_data)\n",
    "\n",
    "print(\"Loss: \", loss)\n",
    "print(\"Accuracy: {:2.2%}\".format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_knIsO-r4pHb"
   },
   "source": [
    "### 导出模型"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FEuMLJA_Xiwo"
   },
   "source": [
    "为了使模型能够将原始字符串作为输入，您将创建一个 Keras `TextVectorization` 层，该层执行与您的自定义预处理函数相同的步骤。由于您已经训练了一个词汇表，可以使用 `TextVectorization.set_vocabulary`（而不是 `TextVectorization.adapt`）来训练一个新词汇表。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:04.654610Z",
     "iopub.status.busy": "2022-08-31T05:41:04.654346Z",
     "iopub.status.idle": "2022-08-31T05:41:04.668245Z",
     "shell.execute_reply": "2022-08-31T05:41:04.667571Z"
    },
    "id": "_ODkRXbk6aHb"
   },
   "outputs": [],
   "source": [
    "preprocess_layer = TextVectorization(\n",
    "    max_tokens=vocab_size,\n",
    "    standardize=tf_text.case_fold_utf8,\n",
    "    split=tokenizer.tokenize,\n",
    "    output_mode='int',\n",
    "    output_sequence_length=MAX_SEQUENCE_LENGTH)\n",
    "\n",
    "preprocess_layer.set_vocabulary(vocab)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:04.671722Z",
     "iopub.status.busy": "2022-08-31T05:41:04.671150Z",
     "iopub.status.idle": "2022-08-31T05:41:04.681472Z",
     "shell.execute_reply": "2022-08-31T05:41:04.680872Z"
    },
    "id": "G-Cvd27y4qwt"
   },
   "outputs": [],
   "source": [
    "export_model = tf.keras.Sequential(\n",
    "    [preprocess_layer, model,\n",
    "     layers.Activation('sigmoid')])\n",
    "\n",
    "export_model.compile(\n",
    "    loss=losses.SparseCategoricalCrossentropy(from_logits=False),\n",
    "    optimizer='adam',\n",
    "    metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:04.685163Z",
     "iopub.status.busy": "2022-08-31T05:41:04.684597Z",
     "iopub.status.idle": "2022-08-31T05:41:10.834907Z",
     "shell.execute_reply": "2022-08-31T05:41:10.834233Z"
    },
    "id": "Pyg0B4zsc-UD"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-08-31 05:41:09.335295: W tensorflow/core/grappler/optimizers/loop_optimizer.cc:907] Skipping loop optimization for Merge node with control input: sequential_4/text_vectorization_2/UnicodeScriptTokenize/Assert_1/AssertGuard/branch_executed/_185\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "      1/Unknown - 6s 6s/step - loss: 0.3060 - accuracy: 0.8438"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "      8/Unknown - 6s 8ms/step - loss: 0.5053 - accuracy: 0.8008"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     15/Unknown - 6s 8ms/step - loss: 0.5065 - accuracy: 0.7969"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     22/Unknown - 6s 8ms/step - loss: 0.4954 - accuracy: 0.8018"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     29/Unknown - 6s 8ms/step - loss: 0.5091 - accuracy: 0.7990"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     36/Unknown - 6s 8ms/step - loss: 0.5039 - accuracy: 0.8025"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     43/Unknown - 6s 8ms/step - loss: 0.5388 - accuracy: 0.7936"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     50/Unknown - 6s 8ms/step - loss: 0.5259 - accuracy: 0.7953"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     57/Unknown - 6s 8ms/step - loss: 0.5261 - accuracy: 0.7974"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     64/Unknown - 6s 8ms/step - loss: 0.5265 - accuracy: 0.7957"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     71/Unknown - 6s 8ms/step - loss: 0.5220 - accuracy: 0.7967"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "     78/Unknown - 6s 8ms/step - loss: 0.5279 - accuracy: 0.7941"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "79/79 [==============================] - 6s 8ms/step - loss: 0.5273 - accuracy: 0.7944\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  0.5273075699806213\n",
      "Accuracy: 79.44%\n"
     ]
    }
   ],
   "source": [
    "# Create a test dataset of raw strings.\n",
    "test_ds = all_labeled_data.take(VALIDATION_SIZE).batch(BATCH_SIZE)\n",
    "test_ds = configure_dataset(test_ds)\n",
    "\n",
    "loss, accuracy = export_model.evaluate(test_ds)\n",
    "\n",
    "print(\"Loss: \", loss)\n",
    "print(\"Accuracy: {:2.2%}\".format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o6Mm0Y9QYQwE"
   },
   "source": [
    "正如预期的那样，编码验证集上的模型和原始验证集上的导出模型的损失和准确率相同。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Stk2BP8GE-qo"
   },
   "source": [
    "### 在新数据上运行推断"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:10.838905Z",
     "iopub.status.busy": "2022-08-31T05:41:10.838449Z",
     "iopub.status.idle": "2022-08-31T05:41:12.935692Z",
     "shell.execute_reply": "2022-08-31T05:41:12.934748Z"
    },
    "id": "-w1fQGJPD2Yh"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-08-31 05:41:12.614755: W tensorflow/core/grappler/optimizers/loop_optimizer.cc:907] Skipping loop optimization for Merge node with control input: sequential_4/text_vectorization_2/UnicodeScriptTokenize/Assert_1/AssertGuard/branch_executed/_185\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Question:  Join'd to th' Ionians with their flowing robes,\n",
      "Predicted label:  1\n",
      "Question:  the allies, and his armour flashed about him so that he seemed to all\n",
      "Predicted label:  2\n",
      "Question:  And with loud clangor of his arms he fell.\n",
      "Predicted label:  0\n"
     ]
    }
   ],
   "source": [
    "inputs = [\n",
    "    \"Join'd to th' Ionians with their flowing robes,\",  # Label: 1\n",
    "    \"the allies, and his armour flashed about him so that he seemed to all\",  # Label: 2\n",
    "    \"And with loud clangor of his arms he fell.\",  # Label: 0\n",
    "]\n",
    "\n",
    "predicted_scores = export_model.predict(inputs)\n",
    "predicted_labels = tf.math.argmax(predicted_scores, axis=1)\n",
    "\n",
    "for input, label in zip(inputs, predicted_labels):\n",
    "  print(\"Question: \", input)\n",
    "  print(\"Predicted label: \", label.numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9eA8TVdnA-3L"
   },
   "source": [
    "## 使用 TensorFlow Datasets (TFDS) 下载更多数据集\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2QFSxfZ3Vqsn"
   },
   "source": [
    "您可以从 [TensorFlow Datasets](https://tensorflow.google.cn/datasets/catalog/overview) 下载更多数据集。\n",
    "\n",
    "在此示例中，您将使用 [IMDB Large Movie Review Dataset](https://tensorflow.google.cn/datasets/catalog/imdb_reviews) 来训练情感分类模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:12.940199Z",
     "iopub.status.busy": "2022-08-31T05:41:12.939721Z",
     "iopub.status.idle": "2022-08-31T05:41:13.754947Z",
     "shell.execute_reply": "2022-08-31T05:41:13.753887Z"
    },
    "id": "NzC65LOaVw0B"
   },
   "outputs": [],
   "source": [
    "# Training set.\n",
    "train_ds = tfds.load(\n",
    "    'imdb_reviews',\n",
    "    split='train[:80%]',\n",
    "    batch_size=BATCH_SIZE,\n",
    "    shuffle_files=True,\n",
    "    as_supervised=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:13.760046Z",
     "iopub.status.busy": "2022-08-31T05:41:13.759272Z",
     "iopub.status.idle": "2022-08-31T05:41:14.295879Z",
     "shell.execute_reply": "2022-08-31T05:41:14.295122Z"
    },
    "id": "XKGkgPBkFh0k"
   },
   "outputs": [],
   "source": [
    "# Validation set.\n",
    "val_ds = tfds.load(\n",
    "    'imdb_reviews',\n",
    "    split='train[80%:]',\n",
    "    batch_size=BATCH_SIZE,\n",
    "    shuffle_files=True,\n",
    "    as_supervised=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BQjf3YZAb5Ne"
   },
   "source": [
    "打印几个样本："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:14.300228Z",
     "iopub.status.busy": "2022-08-31T05:41:14.299962Z",
     "iopub.status.idle": "2022-08-31T05:41:14.701998Z",
     "shell.execute_reply": "2022-08-31T05:41:14.701060Z"
    },
    "id": "Bq1w8MnfWt2C"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Review:  b\"Instead, go to the zoo, buy some peanuts and feed 'em to the monkeys. Monkeys are funny. People with amnesia who don't say much, just sit there with vacant eyes are not all that funny.<br /><br />Black comedy? There isn't a black person in it, and there isn't one funny thing in it either.<br /><br />Walmart buys these things up somehow and puts them on their dollar rack. It's labeled Unrated. I think they took out the topless scene. They may have taken out other stuff too, who knows? All we know is that whatever they took out, isn't there any more.<br /><br />The acting seemed OK to me. There's a lot of unfathomables tho. It's supposed to be a city? It's supposed to be a big lake? If it's so hot in the church people are fanning themselves, why are they all wearing coats?\"\n",
      "Label:  0\n",
      "Review:  b'Well, was Morgan Freeman any more unusual as God than George Burns? This film sure was better than that bore, \"Oh, God\". I was totally engrossed and LMAO all the way through. Carrey was perfect as the out of sorts anchorman wannabe, and Aniston carried off her part as the frustrated girlfriend in her usual well played performance. I, for one, don\\'t consider her to be either ugly or untalented. I think my favorite scene was when Carrey opened up the file cabinet thinking it could never hold his life history. See if you can spot the file in the cabinet that holds the events of his bathroom humor: I was rolling over this one. Well written and even better played out, this comedy will go down as one of this funnyman\\'s best.'\n",
      "Label:  1\n",
      "Review:  b'I remember stumbling upon this special while channel-surfing in 1965. I had never heard of Barbra before. When the show was over, I thought \"This is probably the best thing on TV I will ever see in my life.\" 42 years later, that has held true. There is still nothing so amazing, so honestly astonishing as the talent that was displayed here. You can talk about all the super-stars you want to, this is the most superlative of them all!<br /><br />You name it, she can do it. Comedy, pathos, sultry seduction, ballads, Barbra is truly a story-teller. Her ability to pull off anything she attempts is legendary. But this special was made in the beginning, and helped to create the legend that she quickly became. In spite of rising so far in such a short time, she has fulfilled the promise, revealing more of her talents as she went along. But they are all here from the very beginning. You will not be disappointed in viewing this.'\n",
      "Label:  1\n",
      "Review:  b\"Firstly, I would like to point out that people who have criticised this film have made some glaring errors. Anything that has a rating below 6/10 is clearly utter nonsense.<br /><br />Creep is an absolutely fantastic film with amazing film effects. The actors are highly believable, the narrative thought provoking and the horror and graphical content extremely disturbing. <br /><br />There is much mystique in this film. Many questions arise as the audience are revealed to the strange and freakish creature that makes habitat in the dark rat ridden tunnels. How was 'Craig' created and what happened to him?<br /><br />A fantastic film with a large chill factor. A film with so many unanswered questions and a film that needs to be appreciated along with others like 28 Days Later, The Bunker, Dog Soldiers and Deathwatch.<br /><br />Look forward to more of these fantastic films!!\"\n",
      "Label:  1\n",
      "Review:  b\"I'm sorry but I didn't like this doc very much. I can think of a million ways it could have been better. The people who made it obviously don't have much imagination. The interviews aren't very interesting and no real insight is offered. The footage isn't assembled in a very informative way, either. It's too bad because this is a movie that really deserves spellbinding special features. One thing I'll say is that Isabella Rosselini gets more beautiful the older she gets. All considered, this only gets a '4.'\"\n",
      "Label:  0\n"
     ]
    }
   ],
   "source": [
    "for review_batch, label_batch in val_ds.take(1):\n",
    "  for i in range(5):\n",
    "    print(\"Review: \", review_batch[i].numpy())\n",
    "    print(\"Label: \", label_batch[i].numpy())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q-lVaukyb75k"
   },
   "source": [
    "您现在可以像以前一样预处理数据并训练模型。\n",
    "\n",
    "注：您将对模型使用 `tf.keras.losses.BinaryCrossentropy` 而不是 `tf.keras.losses.SparseCategoricalCrossentropy`，因为这是一个二元分类问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ciz2CxAsZw3Z"
   },
   "source": [
    "### 准备用于训练的数据集"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:14.707036Z",
     "iopub.status.busy": "2022-08-31T05:41:14.706409Z",
     "iopub.status.idle": "2022-08-31T05:41:16.833612Z",
     "shell.execute_reply": "2022-08-31T05:41:16.832805Z"
    },
    "id": "UzT_t9ihZLH4"
   },
   "outputs": [],
   "source": [
    "vectorize_layer = TextVectorization(\n",
    "    max_tokens=VOCAB_SIZE,\n",
    "    output_mode='int',\n",
    "    output_sequence_length=MAX_SEQUENCE_LENGTH)\n",
    "\n",
    "# Make a text-only dataset (without labels), then call `TextVectorization.adapt`.\n",
    "train_text = train_ds.map(lambda text, labels: text)\n",
    "vectorize_layer.adapt(train_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:16.837967Z",
     "iopub.status.busy": "2022-08-31T05:41:16.837299Z",
     "iopub.status.idle": "2022-08-31T05:41:16.841214Z",
     "shell.execute_reply": "2022-08-31T05:41:16.840598Z"
    },
    "id": "zz-Xrd_ZZ4tB"
   },
   "outputs": [],
   "source": [
    "def vectorize_text(text, label):\n",
    "  text = tf.expand_dims(text, -1)\n",
    "  return vectorize_layer(text), label"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:16.844632Z",
     "iopub.status.busy": "2022-08-31T05:41:16.844012Z",
     "iopub.status.idle": "2022-08-31T05:41:16.930828Z",
     "shell.execute_reply": "2022-08-31T05:41:16.930226Z"
    },
    "id": "ycn0Itd6g5aF"
   },
   "outputs": [],
   "source": [
    "train_ds = train_ds.map(vectorize_text)\n",
    "val_ds = val_ds.map(vectorize_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:16.934636Z",
     "iopub.status.busy": "2022-08-31T05:41:16.933960Z",
     "iopub.status.idle": "2022-08-31T05:41:16.938707Z",
     "shell.execute_reply": "2022-08-31T05:41:16.938102Z"
    },
    "id": "jc11jQTlZ5lj"
   },
   "outputs": [],
   "source": [
    "# Configure datasets for performance as before.\n",
    "train_ds = configure_dataset(train_ds)\n",
    "val_ds = configure_dataset(val_ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SQzoYkaGZ82Z"
   },
   "source": [
    "### 创建、配置和训练模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:16.942007Z",
     "iopub.status.busy": "2022-08-31T05:41:16.941794Z",
     "iopub.status.idle": "2022-08-31T05:41:16.981600Z",
     "shell.execute_reply": "2022-08-31T05:41:16.980938Z"
    },
    "id": "B9IOTLkyZ-a7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_5\"\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Layer (type)                Output Shape              Param #   \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=================================================================\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " embedding_2 (Embedding)     (None, None, 64)          640064    \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " conv1d_2 (Conv1D)           (None, None, 64)          20544     \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " global_max_pooling1d_2 (Glo  (None, 64)               0         \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " balMaxPooling1D)                                                \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " dense_3 (Dense)             (None, 1)                 65        \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "                                                                 \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=================================================================\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total params: 660,673\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Trainable params: 660,673\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Non-trainable params: 0\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = create_model(vocab_size=VOCAB_SIZE + 1, num_labels=1)\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:16.988776Z",
     "iopub.status.busy": "2022-08-31T05:41:16.988229Z",
     "iopub.status.idle": "2022-08-31T05:41:16.996984Z",
     "shell.execute_reply": "2022-08-31T05:41:16.996335Z"
    },
    "id": "xLnDs5dhaBAk"
   },
   "outputs": [],
   "source": [
    "model.compile(\n",
    "    loss=losses.BinaryCrossentropy(from_logits=True),\n",
    "    optimizer='adam',\n",
    "    metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:17.000380Z",
     "iopub.status.busy": "2022-08-31T05:41:16.999902Z",
     "iopub.status.idle": "2022-08-31T05:41:22.456810Z",
     "shell.execute_reply": "2022-08-31T05:41:22.455966Z"
    },
    "id": "rq59QpNzaDMa"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/313 [..............................] - ETA: 3:38 - loss: 0.6925 - accuracy: 0.5156"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 14/313 [>.............................] - ETA: 1s - loss: 0.6924 - accuracy: 0.4866  "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 28/313 [=>............................] - ETA: 1s - loss: 0.6916 - accuracy: 0.4944"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 42/313 [===>..........................] - ETA: 1s - loss: 0.6907 - accuracy: 0.5004"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 56/313 [====>.........................] - ETA: 0s - loss: 0.6891 - accuracy: 0.5003"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 70/313 [=====>........................] - ETA: 0s - loss: 0.6868 - accuracy: 0.4920"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 84/313 [=======>......................] - ETA: 0s - loss: 0.6840 - accuracy: 0.4952"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 98/313 [========>.....................] - ETA: 0s - loss: 0.6785 - accuracy: 0.4984"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "112/313 [=========>....................] - ETA: 0s - loss: 0.6712 - accuracy: 0.4990"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "127/313 [===========>..................] - ETA: 0s - loss: 0.6630 - accuracy: 0.5111"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "141/313 [============>.................] - ETA: 0s - loss: 0.6536 - accuracy: 0.5222"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "156/313 [=============>................] - ETA: 0s - loss: 0.6408 - accuracy: 0.5389"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "170/313 [===============>..............] - ETA: 0s - loss: 0.6292 - accuracy: 0.5557"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "184/313 [================>.............] - ETA: 0s - loss: 0.6193 - accuracy: 0.5695"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "199/313 [==================>...........] - ETA: 0s - loss: 0.6056 - accuracy: 0.5862"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "213/313 [===================>..........] - ETA: 0s - loss: 0.5957 - accuracy: 0.5985"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "227/313 [====================>.........] - ETA: 0s - loss: 0.5843 - accuracy: 0.6107"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "241/313 [======================>.......] - ETA: 0s - loss: 0.5740 - accuracy: 0.6227"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "254/313 [=======================>......] - ETA: 0s - loss: 0.5654 - accuracy: 0.6320"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "268/313 [========================>.....] - ETA: 0s - loss: 0.5566 - accuracy: 0.6414"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "282/313 [==========================>...] - ETA: 0s - loss: 0.5497 - accuracy: 0.6493"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "296/313 [===========================>..] - ETA: 0s - loss: 0.5414 - accuracy: 0.6575"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "310/313 [============================>.] - ETA: 0s - loss: 0.5351 - accuracy: 0.6638"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "313/313 [==============================] - 3s 7ms/step - loss: 0.5339 - accuracy: 0.6650 - val_loss: 0.3707 - val_accuracy: 0.8270\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 2/3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/313 [..............................] - ETA: 1s - loss: 0.5132 - accuracy: 0.7188"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 16/313 [>.............................] - ETA: 1s - loss: 0.3685 - accuracy: 0.8291"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 31/313 [=>............................] - ETA: 0s - loss: 0.3754 - accuracy: 0.8311"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 46/313 [===>..........................] - ETA: 0s - loss: 0.3634 - accuracy: 0.8336"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 62/313 [====>.........................] - ETA: 0s - loss: 0.3622 - accuracy: 0.8342"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 77/313 [======>.......................] - ETA: 0s - loss: 0.3611 - accuracy: 0.8346"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 92/313 [=======>......................] - ETA: 0s - loss: 0.3554 - accuracy: 0.8354"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "107/313 [=========>....................] - ETA: 0s - loss: 0.3490 - accuracy: 0.8400"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "122/313 [==========>...................] - ETA: 0s - loss: 0.3484 - accuracy: 0.8417"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "137/313 [============>.................] - ETA: 0s - loss: 0.3437 - accuracy: 0.8445"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "152/313 [=============>................] - ETA: 0s - loss: 0.3363 - accuracy: 0.8491"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "167/313 [===============>..............] - ETA: 0s - loss: 0.3344 - accuracy: 0.8507"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "182/313 [================>.............] - ETA: 0s - loss: 0.3299 - accuracy: 0.8521"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "197/313 [=================>............] - ETA: 0s - loss: 0.3259 - accuracy: 0.8543"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "212/313 [===================>..........] - ETA: 0s - loss: 0.3225 - accuracy: 0.8557"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "227/313 [====================>.........] - ETA: 0s - loss: 0.3173 - accuracy: 0.8581"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "242/313 [======================>.......] - ETA: 0s - loss: 0.3123 - accuracy: 0.8615"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "257/313 [=======================>......] - ETA: 0s - loss: 0.3092 - accuracy: 0.8633"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "272/313 [=========================>....] - ETA: 0s - loss: 0.3070 - accuracy: 0.8641"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "287/313 [==========================>...] - ETA: 0s - loss: 0.3030 - accuracy: 0.8665"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "302/313 [===========================>..] - ETA: 0s - loss: 0.2993 - accuracy: 0.8685"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "313/313 [==============================] - 1s 4ms/step - loss: 0.2980 - accuracy: 0.8690 - val_loss: 0.3181 - val_accuracy: 0.8598\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 3/3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "  1/313 [..............................] - ETA: 1s - loss: 0.2965 - accuracy: 0.8906"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 16/313 [>.............................] - ETA: 1s - loss: 0.2227 - accuracy: 0.9082"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 31/313 [=>............................] - ETA: 0s - loss: 0.2329 - accuracy: 0.9057"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 46/313 [===>..........................] - ETA: 0s - loss: 0.2236 - accuracy: 0.9120"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 61/313 [====>.........................] - ETA: 0s - loss: 0.2249 - accuracy: 0.9111"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 77/313 [======>.......................] - ETA: 0s - loss: 0.2233 - accuracy: 0.9119"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 93/313 [=======>......................] - ETA: 0s - loss: 0.2201 - accuracy: 0.9115"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "109/313 [=========>....................] - ETA: 0s - loss: 0.2166 - accuracy: 0.9133"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "124/313 [==========>...................] - ETA: 0s - loss: 0.2164 - accuracy: 0.9134"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "140/313 [============>.................] - ETA: 0s - loss: 0.2127 - accuracy: 0.9155"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "155/313 [=============>................] - ETA: 0s - loss: 0.2069 - accuracy: 0.9179"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "170/313 [===============>..............] - ETA: 0s - loss: 0.2065 - accuracy: 0.9187"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "185/313 [================>.............] - ETA: 0s - loss: 0.2031 - accuracy: 0.9197"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200/313 [==================>...........] - ETA: 0s - loss: 0.2010 - accuracy: 0.9209"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "215/313 [===================>..........] - ETA: 0s - loss: 0.1975 - accuracy: 0.9222"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "230/313 [=====================>........] - ETA: 0s - loss: 0.1944 - accuracy: 0.9238"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "245/313 [======================>.......] - ETA: 0s - loss: 0.1915 - accuracy: 0.9252"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "260/313 [=======================>......] - ETA: 0s - loss: 0.1901 - accuracy: 0.9259"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "275/313 [=========================>....] - ETA: 0s - loss: 0.1877 - accuracy: 0.9270"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "290/313 [==========================>...] - ETA: 0s - loss: 0.1851 - accuracy: 0.9284"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "305/313 [============================>.] - ETA: 0s - loss: 0.1829 - accuracy: 0.9298"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "313/313 [==============================] - 1s 4ms/step - loss: 0.1819 - accuracy: 0.9305 - val_loss: 0.3239 - val_accuracy: 0.8620\n"
     ]
    }
   ],
   "source": [
    "history = model.fit(train_ds, validation_data=val_ds, epochs=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:22.460443Z",
     "iopub.status.busy": "2022-08-31T05:41:22.460154Z",
     "iopub.status.idle": "2022-08-31T05:41:22.686238Z",
     "shell.execute_reply": "2022-08-31T05:41:22.685399Z"
    },
    "id": "gCMWCEtyaEbR"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      " 1/79 [..............................] - ETA: 4s - loss: 0.1777 - accuracy: 0.9062"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "26/79 [========>.....................] - ETA: 0s - loss: 0.3246 - accuracy: 0.8612"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "51/79 [==================>...........] - ETA: 0s - loss: 0.3192 - accuracy: 0.8627"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "77/79 [============================>.] - ETA: 0s - loss: 0.3225 - accuracy: 0.8624"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "79/79 [==============================] - 0s 2ms/step - loss: 0.3239 - accuracy: 0.8620\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loss:  0.32394784688949585\n",
      "Accuracy: 86.20%\n"
     ]
    }
   ],
   "source": [
    "loss, accuracy = model.evaluate(val_ds)\n",
    "\n",
    "print(\"Loss: \", loss)\n",
    "print(\"Accuracy: {:2.2%}\".format(accuracy))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jGtqLXVnaaFy"
   },
   "source": [
    "### 导出模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:22.690294Z",
     "iopub.status.busy": "2022-08-31T05:41:22.689667Z",
     "iopub.status.idle": "2022-08-31T05:41:22.742593Z",
     "shell.execute_reply": "2022-08-31T05:41:22.741866Z"
    },
    "id": "yE9WZARZaZr1"
   },
   "outputs": [],
   "source": [
    "export_model = tf.keras.Sequential(\n",
    "    [vectorize_layer, model,\n",
    "     layers.Activation('sigmoid')])\n",
    "\n",
    "export_model.compile(\n",
    "    loss=losses.SparseCategoricalCrossentropy(from_logits=False),\n",
    "    optimizer='adam',\n",
    "    metrics=['accuracy'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-08-31T05:41:22.747169Z",
     "iopub.status.busy": "2022-08-31T05:41:22.746514Z",
     "iopub.status.idle": "2022-08-31T05:41:22.897966Z",
     "shell.execute_reply": "2022-08-31T05:41:22.897115Z"
    },
    "id": "bhF8tDH-afoC"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Question:  This is a fantastic movie.\n",
      "Predicted label:  1\n",
      "Question:  This is a bad movie.\n",
      "Predicted label:  0\n",
      "Question:  This movie was so bad that it was good.\n",
      "Predicted label:  0\n",
      "Question:  I will never say yes to watching this movie.\n",
      "Predicted label:  0\n"
     ]
    }
   ],
   "source": [
    "# 0 --> negative review\n",
    "# 1 --> positive review\n",
    "inputs = [\n",
    "    \"This is a fantastic movie.\",\n",
    "    \"This is a bad movie.\",\n",
    "    \"This movie was so bad that it was good.\",\n",
    "    \"I will never say yes to watching this movie.\",\n",
    "]\n",
    "\n",
    "predicted_scores = export_model.predict(inputs)\n",
    "predicted_labels = [int(round(x[0])) for x in predicted_scores]\n",
    "\n",
    "for input, label in zip(inputs, predicted_labels):\n",
    "  print(\"Question: \", input)\n",
    "  print(\"Predicted label: \", label)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q1KSXDFPWiPN"
   },
   "source": [
    "## 结论\n",
    "\n",
    "本教程演示了几种加载和预处理文本的方法。接下来，您可以探索其他文本预处理 [TensorFlow Text](https://tensorflow.google.cn/text) 教程，例如：\n",
    "\n",
    "- [使用 TF Text 进行 BERT 预处理](https://tensorflow.google.cn/text/guide/bert_preprocessing_guide)\n",
    "- [使用 TF Text 进行词例化](https://tensorflow.google.cn/text/guide/tokenizers)\n",
    "- [子词分词器](https://tensorflow.google.cn/text/guide/subwords_tokenizer)\n",
    "\n",
    "此外，您还可以在 [TensorFlow Datasets](https://tensorflow.google.cn/datasets/catalog/overview) 上找到新的数据集。而且，要详细了解 `tf.data`，请查看有关[构建输入流水线](../../guide/data.ipynb)的指南。"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "text.ipynb",
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
